-- Logs begin at Wed 2022-10-12 13:58:40 UTC, end at Wed 2022-10-12 14:40:44 UTC. -- Oct 12 14:25:22.963975 dv-cinder-812685-17-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 12 14:25:22.971340 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 12 14:25:22.973321 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Oct 12 14:25:22 2022] *** Oct 12 14:25:22.973321 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 12 14:25:22.973321 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 12 14:25:22.973321 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: nodename: dv-cinder-812685-17-iscsi Oct 12 14:25:22.973321 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: machine: x86_64 Oct 12 14:25:22.973321 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: clock source: unix Oct 12 14:25:22.973321 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: pcre jit disabled Oct 12 14:25:22.973764 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: detected number of CPU cores: 4 Oct 12 14:25:22.973764 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: current working directory: / Oct 12 14:25:22.973764 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: detected binary path: /usr/bin/uwsgi-core Oct 12 14:25:22.973764 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: your processes number limit is 47680 Oct 12 14:25:22.973764 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: your memory page size is 4096 bytes Oct 12 14:25:22.973764 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: detected max file descriptor number: 2048 Oct 12 14:25:22.973764 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: lock engine: pthread robust mutexes Oct 12 14:25:22.973764 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: thunder lock: enabled Oct 12 14:25:22.973764 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 12 14:25:22.973764 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 12 14:25:23.014359 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: Python main interpreter initialized at 0x558051bb89c0 Oct 12 14:25:23.014359 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: python threads support enabled Oct 12 14:25:23.014359 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: your server socket listen backlog is limited to 100 connections Oct 12 14:25:23.014359 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: your mercy for graceful operations on workers is 90 seconds Oct 12 14:25:23.014744 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: mapped 403077 bytes (393 KB) for 2 cores Oct 12 14:25:23.014825 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: *** Operational MODE: preforking *** Oct 12 14:25:23.015077 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: *** uWSGI is running in multiple interpreter mode *** Oct 12 14:25:23.015077 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: spawned uWSGI master process (pid: 81552) Oct 12 14:25:23.015294 dv-cinder-812685-17-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 12 14:25:23.016336 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: spawned uWSGI worker 1 (pid: 81565, cores: 1) Oct 12 14:25:23.019581 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: spawned uWSGI worker 2 (pid: 81566, cores: 1) Oct 12 14:25:23.019581 dv-cinder-812685-17-iscsi devstack@keystone.service[81552]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 12 14:25:27.307920 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 14:25:27.307920 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: from cryptography.utils import int_from_bytes Oct 12 14:25:27.308718 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 14:25:27.308718 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: from cryptography.utils import int_from_bytes Oct 12 14:25:27.308837 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 14:25:27.308837 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: from cryptography.utils import int_from_bytes Oct 12 14:25:27.308837 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 14:25:27.308837 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: from cryptography.utils import int_from_bytes Oct 12 14:25:27.358641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 12 14:25:27.358790 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 12 14:25:27.358888 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 12 14:25:27.358960 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 12 14:25:27.359021 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] command line args: [] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 12 14:25:27.359078 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] command line args: [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 12 14:25:27.359286 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 12 14:25:27.359388 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 12 14:25:27.359468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ================================================================================ {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 12 14:25:27.359546 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ================================================================================ {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 12 14:25:27.359699 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] admin_token = **** {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.359787 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] admin_token = **** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.359942 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] config_dir = [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.360021 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] config_dir = [] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.360125 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.360210 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.360307 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] config_source = [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.360390 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] config_source = [] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.360472 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] debug = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.360544 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] debug = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.360671 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.362064 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.362168 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.362236 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.362236 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.362349 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.362412 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.362470 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.362536 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.362536 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] list_limit = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.362642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.362704 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] log_config_append = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.362761 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] list_limit = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.362817 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.362883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] log_config_append = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.362946 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] log_dir = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.363009 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.363066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] log_file = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.363126 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] log_dir = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.363225 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.363298 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] log_file = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.363298 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.363401 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.363463 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.363525 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.363587 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.363676 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.363741 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.363798 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.363900 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.364028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.364105 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.364174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.364380 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.364473 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.364575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.364732 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.364874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.364999 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.365122 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.365245 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.365368 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.365501 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.365632 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.365715 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.365833 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.365907 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.366027 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.366103 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.366229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.366443 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.366625 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] publish_errors = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.366703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.366804 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] publish_errors = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.366886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.367035 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.367188 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.367320 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.367441 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.367441 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.367660 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.367792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.367916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.368044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.368161 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] standard_threads = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.368275 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] standard_threads = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.368388 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.368515 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.368631 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.368631 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.368850 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.368965 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] use_journal = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.369079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.369242 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] use_json = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.369360 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] use_journal = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.369480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] use_stderr = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.369597 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] use_json = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.369711 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] use_syslog = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.369826 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] use_stderr = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.369826 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] use_syslog = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.370049 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.370164 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 14:25:27.370283 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.370399 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.370520 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.370645 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.370767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.370888 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.370993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.371107 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.371242 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.371369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.371492 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.371614 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.371730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.371844 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.371964 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.372091 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.372213 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.372329 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.372442 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.372557 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.372660 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.372782 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.372899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.373019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.373134 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.373248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.373362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.373485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.373606 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.373722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.373844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.373961 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.374075 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.374201 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.374318 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.374432 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.374552 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.374668 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.374861 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.374979 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.375100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.375240 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.375355 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.375480 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.375608 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.375731 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.375848 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.375977 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.376093 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.376214 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.376341 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.376458 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.376590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.376710 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.376808 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.376904 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.377008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.377113 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.377236 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.377355 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.377469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.377583 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.377709 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.377825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.377940 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.378060 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.378175 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.378295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.378419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.378535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.378655 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.378776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.378890 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.379005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.379126 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.379313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.379429 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.379543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.379657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.379772 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.379902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.380019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.380133 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.380253 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.380368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.380483 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.380670 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.380788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.380902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.381016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.381131 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.381246 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.381369 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.381485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.381600 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.381715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.381829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.381943 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.382049 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.382148 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.382244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.382339 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.382435 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.382532 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.382636 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.382734 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.382829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.382932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.383050 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.383192 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.383311 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.383426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.383546 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.383714 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.383834 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.383959 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.384076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.384232 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.384352 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.384467 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.384581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.384704 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.384820 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.384931 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.385046 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.385170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.385301 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.385428 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.385545 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.385673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.385796 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.385913 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.386017 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.386137 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.386262 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.386392 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.386510 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.386626 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.386746 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.386858 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.386974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.387101 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.387274 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.387393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.387518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.387635 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.387750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.387874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.387991 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.388107 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.388222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.388337 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.388452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.388576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.388693 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.388808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.388922 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.389037 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.389151 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.389274 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.389391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.389507 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.389622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.389737 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.389897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.390023 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.390140 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.390262 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.390384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.390500 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.390622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.390747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.390865 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.390980 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.391095 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.391225 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.391340 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.391467 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.391586 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.391701 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.391816 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.391936 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.392053 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.392749 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.392870 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.392991 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.393108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.393379 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.393519 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.393646 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.393776 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.393893 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.394009 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.394123 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.394238 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.394353 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.394476 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.394595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.394711 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.394832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.394949 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.395074 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.395219 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.395219 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.395440 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.395555 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.395670 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.395786 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.395913 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.395913 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.396139 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.396139 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.396357 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.396498 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] auth.external = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.396629 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.396754 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.396882 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] auth.external = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.396882 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.397101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.397101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.397330 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.397449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] auth.password = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.397574 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.397698 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] auth.token = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.397815 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] auth.password = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.397932 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.398047 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] auth.token = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.398162 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.398286 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.398286 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.398504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.398504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.398720 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.398720 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.398955 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.399073 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.399214 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.399332 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.399449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.399575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.399702 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] credential.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.399826 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] credential.caching = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.399944 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.400061 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.400176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.400292 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.400416 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.400534 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.400651 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.400773 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.400890 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.401007 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.401131 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.401249 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.401371 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.401489 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.401605 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.401720 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.401844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.401970 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.402090 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.402205 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.402321 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.402436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.402567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.402686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.402808 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.402925 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.403040 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.403185 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.403306 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.403423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.403539 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] federation.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.403655 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] federation.caching = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.403771 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.403898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.404016 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.404139 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.404257 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.404374 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.404496 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.404628 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.404747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.404864 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.404980 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.405096 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.405213 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.405338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.405456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.405572 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.405688 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.405804 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.405927 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.406052 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.406179 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.406303 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.406420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.406535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity.caching = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.406651 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.407103 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.407321 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.407441 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.407557 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.407673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.407822 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.408619 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.408747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.408857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.408973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.409089 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.409205 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.409329 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.409449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.409565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.409681 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.409797 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.409913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.410037 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.410163 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.410281 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.410454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.410575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.410691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.410816 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.410934 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.411050 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.411183 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.411294 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.411409 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.411539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.411658 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.411774 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.411890 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.412005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.412120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.412120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.412348 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.412463 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.412579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.412694 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.412808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.412931 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.413048 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.413163 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.413278 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.413393 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.413516 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.413636 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.413751 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.413866 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.413988 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.414105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.414229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.414346 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.414462 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.414577 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.414692 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.414807 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.414930 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.415048 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.415239 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.415359 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.415475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.415591 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.415720 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.415838 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.415953 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.416068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.416184 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.416299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.416422 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.416540 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.416655 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.416771 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.416886 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.417001 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.417127 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.417127 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.417345 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.417465 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.417580 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.417695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.417819 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.417819 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.418039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.418162 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.418279 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.418279 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.418504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.418622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.418737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.418737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.418955 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.419070 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.419210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.419330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.419446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.419561 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.419677 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.419792 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.419916 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.419916 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.420135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.420135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.420351 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.420351 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.420577 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.420695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.420810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.420926 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.421048 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.421165 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.421288 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.421288 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.421514 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.421514 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.421730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.421846 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.421971 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.422088 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.422203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.422318 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.422434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.422549 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.422676 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.422799 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.422917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.423033 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.423178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.423297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.423426 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.423545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.423643 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.423759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.423903 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.424069 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.424201 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.424320 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.424437 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.424554 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.424671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.424787 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.424911 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.425030 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.425147 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.425263 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.425380 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.425496 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.425620 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.425738 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.425855 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.425971 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.426087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.426204 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.426329 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.426447 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.426564 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.426680 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.426797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.426912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.427037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.427182 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.427307 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.427307 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.427526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.427642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.427771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.427890 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.428006 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.428122 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.428238 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.428354 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] resource.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.428478 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.428596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.428711 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] resource.caching = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.428827 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.428942 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.429066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.429184 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.429301 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.429417 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.429532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.429648 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.429776 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.429895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.430011 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.430127 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.430243 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.430360 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.430554 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.430673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.430790 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] role.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.430906 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.431021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] role.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.431137 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] role.caching = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.431287 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.431407 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] role.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.431523 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.431638 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.431754 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.431871 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.431995 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.432113 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.432229 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.432345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.432462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.432578 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.432702 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.432821 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.432937 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.433054 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.433169 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.433285 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.433410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.433529 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.433652 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.433775 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.433892 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.434008 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.434132 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.434251 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.434367 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.434483 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.434483 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.434702 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.434702 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.434929 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.435045 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.435211 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.435331 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.435460 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.435579 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.435695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.435811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.435927 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.436042 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.436167 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.436285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.436402 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.436517 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.436633 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.436874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.436992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.437108 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.437224 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.437344 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.437460 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.437583 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.437702 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.437819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.437934 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.438050 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.438165 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.438289 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.438407 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.438523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.438639 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.438755 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.438869 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.438993 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.439125 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] token.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.439302 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.439420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.439537 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] token.caching = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.439653 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.439782 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.439902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.440019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.440135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.440251 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.440367 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.440491 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.440609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.440726 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.440842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.440964 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.441082 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.441207 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.441325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.441441 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.441557 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.441673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.441789 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.441915 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.442032 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.442149 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.442265 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.442381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.442381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.442606 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.442606 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.442834 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.442961 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.443079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.443079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.443335 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.443453 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.443569 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.connection = **** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.443569 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.443796 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.connection = **** {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.443912 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.444028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.444144 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.444259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.444383 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.444500 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.444615 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.444731 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.444846 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.444944 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.445051 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.445169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.445284 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.445399 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.445515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.445631 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.445755 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.445873 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.445990 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.446107 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.446278 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.446398 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.446523 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.446641 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.446758 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.446874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.446991 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.447107 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.447260 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.447381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.447497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.447613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.447729 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.447844 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.447969 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 12 14:25:27.448088 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 14:25:27.448088 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 12 14:25:27.473371 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.473583 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.473688 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.473812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.473894 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.474032 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.482104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.482308 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.510049 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.510174 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.510305 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.510386 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.564444 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.564591 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.613922 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.614145 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.614278 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.614514 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.770879 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.771060 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.771219 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.771334 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.771444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.771549 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.771631 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.771766 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.771848 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.771979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.772062 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.772190 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.795547 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.795697 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.795784 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.795932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.796015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.796156 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.796238 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.796378 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.796470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.796596 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.796679 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.796810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.796890 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.797023 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.808060 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.808237 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.821422 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.821535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.821687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.821796 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.821879 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.821987 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.822104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.822188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.822324 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.822408 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.822540 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.822623 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.822753 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.822843 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 14:25:27.839625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.839831 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 12 14:25:27.840139 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.840222 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.840345 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 12 14:25:27.840442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 12 14:25:27.840613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.840694 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.840860 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 12 14:25:27.840940 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.841118 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.841200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.841372 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.841452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.841632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.841890 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.851568 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.851844 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.852112 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.852377 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.852459 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.852652 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.852923 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.853204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.859460 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.860257 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.862454 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.862731 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.863008 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.863314 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.863399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.863597 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.863695 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.863855 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.863936 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.864127 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.864209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.864399 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.864482 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.864664 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.864744 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.864933 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.865036 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.865196 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.865276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.865462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.865734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.866001 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.888414 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.889169 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.889882 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.890161 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.890640 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.890924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.894947 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.895262 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.895547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.895909 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.896193 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.896476 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.901869 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.902153 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.902829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.903111 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.908552 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.908843 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.909127 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.909406 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.909512 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.909679 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.909804 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.909938 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.910088 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.910203 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.910372 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.910454 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.910648 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.910925 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.911219 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.911488 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.925998 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.926273 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.927014 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.927317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.930138 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.931179 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.933392 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.933687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.934416 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.934710 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.937914 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.939498 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.941718 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.942695 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.944973 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.945272 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.945556 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.945841 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.945958 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.946137 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.946277 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.946404 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.946529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.946811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.947088 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.947398 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.962263 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.962552 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.962820 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.963085 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.963317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.963604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.963879 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.964142 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.969929 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.971527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.972200 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.973066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.973733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.974008 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.974599 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.974876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.976690 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.976971 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.977247 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.977526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.977607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.977857 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.977939 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.978136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.978418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.978807 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.989298 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.989583 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.989862 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.990034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:27.990128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.990317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.990505 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.990587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.990782 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.990865 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.991071 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.991247 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.991528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:27.991818 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.007111 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:28.007737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:28.010040 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:28.010640 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:28.013038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.013569 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.014539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.015031 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.016041 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:28.016326 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.016536 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:28.016730 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.016859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.017753 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.023737 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.024178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.025199 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:28.025646 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:28.028132 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.028527 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.028609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.028813 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.028965 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.029089 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.029252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.029531 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.037336 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:28.037626 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.037718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:28.037908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.038002 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.038190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.044789 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:28.045074 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.045156 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 14:25:28.045353 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.045435 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.045638 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.045719 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.045912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.045996 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.046177 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.046278 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.046451 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.046546 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.046720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.046811 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.046984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.047061 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.047278 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.047362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.047556 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.047638 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.047840 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.047924 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.048147 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.048265 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.048638 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.048725 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.048814 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 14:25:28.083857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558051bb89c0 pid: 81566 (default app) Oct 12 14:25:28.083973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558051bb89c0 pid: 81565 (default app) Oct 12 14:25:28.087028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d7491-b0d2-4132-aff3-3c210a9631a6 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:28.087337 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d7491-b0d2-4132-aff3-3c210a9631a6 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:28.087610 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d7491-b0d2-4132-aff3-3c210a9631a6 None None] PATH_INFO: `/v3` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:28.091698 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Wed Oct 12 14:25:28 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:34.853317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03717bf2-28ce-44de-a801-b41550284362 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:34.853898 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03717bf2-28ce-44de-a801-b41550284362 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:34.853898 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03717bf2-28ce-44de-a801-b41550284362 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:34.858026 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:25:34.864031 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8296f2-1764-4e20-a75d-6637e54b8c20 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:34.864564 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8296f2-1764-4e20-a75d-6637e54b8c20 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:34.864564 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8296f2-1764-4e20-a75d-6637e54b8c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:34.934086 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: INFO dbcounter [None req-7b8296f2-1764-4e20-a75d-6637e54b8c20 None None] Registered counter for database keystone Oct 12 14:25:34.953332 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG oslo_db.sqlalchemy.engines [None req-7b8296f2-1764-4e20-a75d-6637e54b8c20 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81566) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 14:25:34.959543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writer thread running {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 14:25:35.137129 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG passlib.handlers.bcrypt [None req-7b8296f2-1764-4e20-a75d-6637e54b8c20 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81566) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 12 14:25:35.137430 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG passlib.handlers.bcrypt [None req-7b8296f2-1764-4e20-a75d-6637e54b8c20 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81566) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 12 14:25:35.197344 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-7b8296f2-1764-4e20-a75d-6637e54b8c20 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:35.198949 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-7b8296f2-1764-4e20-a75d-6637e54b8c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:35.326776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 12 14:25:35.629727 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03717bf2-28ce-44de-a801-b41550284362 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:35.699092 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: INFO dbcounter [None req-03717bf2-28ce-44de-a801-b41550284362 None None] Registered counter for database keystone Oct 12 14:25:35.721241 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG oslo_db.sqlalchemy.engines [None req-03717bf2-28ce-44de-a801-b41550284362 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81565) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 14:25:35.727561 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writer thread running {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 14:25:35.848795 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03717bf2-28ce-44de-a801-b41550284362 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:35.852865 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07ce73-be37-46b3-b528-910ea1fb50b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:35.853478 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07ce73-be37-46b3-b528-910ea1fb50b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:35.853998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07ce73-be37-46b3-b528-910ea1fb50b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:35.853998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07ce73-be37-46b3-b528-910ea1fb50b1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:35.862266 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07ce73-be37-46b3-b528-910ea1fb50b1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:35.915636 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1c07ce73-be37-46b3-b528-910ea1fb50b1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 12 14:25:35.915973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07ce73-be37-46b3-b528-910ea1fb50b1 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:35.925348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Wed Oct 12 14:25:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 12 14:25:35.931276 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8296f2-1764-4e20-a75d-6637e54b8c20 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:35.947763 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8296f2-1764-4e20-a75d-6637e54b8c20 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:35.952055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2d106-8a5d-4169-bf6f-214d09311b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:35.953139 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2d106-8a5d-4169-bf6f-214d09311b6e admin admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:35.953937 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2d106-8a5d-4169-bf6f-214d09311b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:35.954204 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2d106-8a5d-4169-bf6f-214d09311b6e admin admin] PATH_INFO: `/v3/projects/671bb1b187584e1b95d55373780c0171` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:35.955940 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2d106-8a5d-4169-bf6f-214d09311b6e admin admin] RBAC: Authorizing `identity:get_project(project_id=671bb1b187584e1b95d55373780c0171)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:36.009420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2fb2d106-8a5d-4169-bf6f-214d09311b6e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 12 14:25:36.009758 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2d106-8a5d-4169-bf6f-214d09311b6e admin admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:36.014166 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Wed Oct 12 14:25:35 2022] GET /identity/v3/projects/671bb1b187584e1b95d55373780c0171 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:38.121538 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef256882-0cd9-4a6e-b603-e8bb1b4e68b1 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:38.122073 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef256882-0cd9-4a6e-b603-e8bb1b4e68b1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:38.122073 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef256882-0cd9-4a6e-b603-e8bb1b4e68b1 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:38.126336 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:25:38.131942 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f5512-ccf6-4d3d-9970-b79a67858f4e None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:38.132365 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f5512-ccf6-4d3d-9970-b79a67858f4e None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:38.132482 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f5512-ccf6-4d3d-9970-b79a67858f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:38.152612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-ff5f5512-ccf6-4d3d-9970-b79a67858f4e None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:38.154102 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-ff5f5512-ccf6-4d3d-9970-b79a67858f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:38.162553 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 14:25:38.433989 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef256882-0cd9-4a6e-b603-e8bb1b4e68b1 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:38.446730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef256882-0cd9-4a6e-b603-e8bb1b4e68b1 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:38.450954 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d03a4-cc15-4b07-8a6f-ba6c7d3d9196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:38.451587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d03a4-cc15-4b07-8a6f-ba6c7d3d9196 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:38.451866 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d03a4-cc15-4b07-8a6f-ba6c7d3d9196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:38.452137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d03a4-cc15-4b07-8a6f-ba6c7d3d9196 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:38.460356 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d03a4-cc15-4b07-8a6f-ba6c7d3d9196 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:38.462115 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d03a4-cc15-4b07-8a6f-ba6c7d3d9196 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:38.470734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Wed Oct 12 14:25:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 14:25:38.476083 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f5512-ccf6-4d3d-9970-b79a67858f4e None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:38.485209 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f5512-ccf6-4d3d-9970-b79a67858f4e None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:38.489469 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2cdb4-a061-4a27-b9b1-a3f67795effd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:38.490083 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2cdb4-a061-4a27-b9b1-a3f67795effd admin admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:38.490357 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2cdb4-a061-4a27-b9b1-a3f67795effd admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:38.490627 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2cdb4-a061-4a27-b9b1-a3f67795effd admin admin] PATH_INFO: `/v3/users/3ce73a2078a24b19af860a9906b4d83e` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:38.491943 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2cdb4-a061-4a27-b9b1-a3f67795effd admin admin] RBAC: Authorizing `identity:get_user(user_id=3ce73a2078a24b19af860a9906b4d83e)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:38.493749 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2cdb4-a061-4a27-b9b1-a3f67795effd admin admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:38.495272 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Wed Oct 12 14:25:38 2022] GET /identity/v3/users/3ce73a2078a24b19af860a9906b4d83e => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:40.689907 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc6ca7-a30e-4eda-8ca9-6a399535dd6a None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:40.690691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc6ca7-a30e-4eda-8ca9-6a399535dd6a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:40.690854 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc6ca7-a30e-4eda-8ca9-6a399535dd6a None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:40.692285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:25:40.697608 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6773b66-87be-4f53-8f4f-822c3de9b0af None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:40.698017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6773b66-87be-4f53-8f4f-822c3de9b0af None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:40.698253 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6773b66-87be-4f53-8f4f-822c3de9b0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:40.716902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-d6773b66-87be-4f53-8f4f-822c3de9b0af None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:40.718079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-d6773b66-87be-4f53-8f4f-822c3de9b0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:40.746722 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c744f8-196e-4222-9e4c-ac6b18cfb266 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:40.747001 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c744f8-196e-4222-9e4c-ac6b18cfb266 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:40.747301 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c744f8-196e-4222-9e4c-ac6b18cfb266 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:40.748608 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:25:40.754458 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05bf2c-1cd2-477f-adc6-efe37c45dc11 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:40.754740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05bf2c-1cd2-477f-adc6-efe37c45dc11 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:40.755010 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05bf2c-1cd2-477f-adc6-efe37c45dc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:40.797980 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG passlib.handlers.bcrypt [None req-ff05bf2c-1cd2-477f-adc6-efe37c45dc11 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81565) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 12 14:25:40.798464 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG passlib.handlers.bcrypt [None req-ff05bf2c-1cd2-477f-adc6-efe37c45dc11 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81565) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 12 14:25:40.799495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:25:40.848892 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ff05bf2c-1cd2-477f-adc6-efe37c45dc11 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:40.850410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ff05bf2c-1cd2-477f-adc6-efe37c45dc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:40.949819 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:25:41.065539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6773b66-87be-4f53-8f4f-822c3de9b0af None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:41.089047 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:41.168758 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6773b66-87be-4f53-8f4f-822c3de9b0af None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:41.200876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07efbd-7240-4fd7-8bfb-17cfdb5308e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:41.201592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07efbd-7240-4fd7-8bfb-17cfdb5308e7 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:41.201871 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07efbd-7240-4fd7-8bfb-17cfdb5308e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:41.202141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07efbd-7240-4fd7-8bfb-17cfdb5308e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:41.213555 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07efbd-7240-4fd7-8bfb-17cfdb5308e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:25:41.219944 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05bf2c-1cd2-477f-adc6-efe37c45dc11 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:41.222695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-ee07efbd-7240-4fd7-8bfb-17cfdb5308e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:25:41.222695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:25:41.222695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:25:41.222695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:25:41.222695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:25:41.222695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:25:41.222695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:25:41.222695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:25:41.222695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:25:41.222695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:25:41.222695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:25:41.222695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:25:41.222695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:25:41.222695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:25:41.222695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:25:41.222695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:25:41.223576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:25:41.223576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:25:41.223576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:25:41.223576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:25:41.223576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:25:41.223576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:25:41.223576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:25:41.223576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:25:41.223576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:25:41.223576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:25:41.223576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:25:41.223576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:25:41.223576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:25:41.223576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:25:41.223576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:25:41.223576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:25:41.224898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:25:41.224898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:25:41.224898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:25:41.224898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:25:41.224898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:25:41.224898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:25:41.224898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:25:41.224898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:25:41.224898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:25:41.224898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:25:41.224898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:25:41.224898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:25:41.224898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:25:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:25:41.229431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07efbd-7240-4fd7-8bfb-17cfdb5308e7 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:41.309909 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:41.324205 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05bf2c-1cd2-477f-adc6-efe37c45dc11 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:41.326423 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:41.329034 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07efbd-7240-4fd7-8bfb-17cfdb5308e7 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:41.361482 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6a47b-7899-4186-864e-824d5a3b7edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:41.362345 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6a47b-7899-4186-864e-824d5a3b7edb None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:41.362454 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424da731-35c7-4ea4-b6de-1875dee3ba0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:41.362623 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6a47b-7899-4186-864e-824d5a3b7edb None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:41.362892 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6a47b-7899-4186-864e-824d5a3b7edb None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:41.363050 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424da731-35c7-4ea4-b6de-1875dee3ba0b None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:41.363360 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424da731-35c7-4ea4-b6de-1875dee3ba0b None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:41.363635 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424da731-35c7-4ea4-b6de-1875dee3ba0b None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:41.363899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6a47b-7899-4186-864e-824d5a3b7edb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:41.364303 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424da731-35c7-4ea4-b6de-1875dee3ba0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:25:41.364604 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424da731-35c7-4ea4-b6de-1875dee3ba0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:41.365334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6a47b-7899-4186-864e-824d5a3b7edb None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:41.365988 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424da731-35c7-4ea4-b6de-1875dee3ba0b None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:41.377283 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.sql.core [None req-25f6a47b-7899-4186-864e-824d5a3b7edb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 14:25:41.377283 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 14:25:41.377283 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [parameters: {'id': '3932f7ca5bae4b3fb9ad13b6afba991f', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 12 14:25:41.377283 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81565) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 14:25:41.378989 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:25:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:25:41.384703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424da731-35c7-4ea4-b6de-1875dee3ba0b None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:41.395957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-25f6a47b-7899-4186-864e-824d5a3b7edb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 12 14:25:41.395957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:25:41.395957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 14:25:41.395957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 14:25:41.395957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 14:25:41.395957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 14:25:41.395957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 14:25:41.395957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 14:25:41.395957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 14:25:41.395957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application conn.query(q) Oct 12 14:25:41.395957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 14:25:41.395957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 14:25:41.395957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 14:25:41.395957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result.read() Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 12 14:25:41.396998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application next(self.gen) Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application yield resource Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application next(self.gen) Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise self.value Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application session.commit() Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 14:25:41.398348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.session.flush() Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application compat.raise_( Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 14:25:41.399399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 14:25:41.400446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 14:25:41.400446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 14:25:41.400446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 14:25:41.400446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 12 14:25:41.400446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 12 14:25:41.400446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 14:25:41.400446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 14:25:41.400446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 14:25:41.400446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 14:25:41.400446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 14:25:41.400446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 14:25:41.400446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 14:25:41.400446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 14:25:41.400446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 14:25:41.400446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application conn.query(q) Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result.read() Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 14:25:41.401333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 14:25:41.402486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 14:25:41.402486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 14:25:41.402486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 14:25:41.402486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 14:25:41.402486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 14:25:41.402486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 14:25:41.402486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 14:25:41.402486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [parameters: {'id': '3932f7ca5bae4b3fb9ad13b6afba991f', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 12 14:25:41.402486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 14:25:41.402486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:25:41.402486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:25:41.402486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:25:41.402486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:25:41.402486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 12 14:25:41.402486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:25:41.403438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:25:41.404418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 12 14:25:41.404418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 12 14:25:41.404418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:25:41.404418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:25:41.404418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 12 14:25:41.404418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 12 14:25:41.404418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:25:41.404418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:25:41.404418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 12 14:25:41.404418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 12 14:25:41.404418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 12 14:25:41.404418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:25:41.404418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 8/16] 10.222.0.16 () {60 vars in 1137 bytes} [Wed Oct 12 14:25:41 2022] POST /identity/v3/domains => generated 190 bytes in 178 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:25:41.404418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6a47b-7899-4186-864e-824d5a3b7edb None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:41.497031 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424da731-35c7-4ea4-b6de-1875dee3ba0b None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:41.505767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6a47b-7899-4186-864e-824d5a3b7edb None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:41.537277 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0b908-b296-48e6-b901-0f25e55cb683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:41.537914 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0b908-b296-48e6-b901-0f25e55cb683 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:41.538194 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0b908-b296-48e6-b901-0f25e55cb683 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:41.538470 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0b908-b296-48e6-b901-0f25e55cb683 None admin] PATH_INFO: `/v3/users/3ce73a2078a24b19af860a9906b4d83e` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:41.539645 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f0b908-b296-48e6-b901-0f25e55cb683 None admin] RBAC: Authorizing `identity:get_user(user_id=3ce73a2078a24b19af860a9906b4d83e)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:41.541140 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f0b908-b296-48e6-b901-0f25e55cb683 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:41.542103 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757eaf0-0b37-4dce-8ca5-203b4f79dfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:41.542701 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:25:41 2022] GET /identity/v3/users/3ce73a2078a24b19af860a9906b4d83e => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:25:41.542967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757eaf0-0b37-4dce-8ca5-203b4f79dfb2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:41.543282 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757eaf0-0b37-4dce-8ca5-203b4f79dfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:41.543556 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757eaf0-0b37-4dce-8ca5-203b4f79dfb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:41.548465 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0b908-b296-48e6-b901-0f25e55cb683 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:41.563595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:41.563934 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757eaf0-0b37-4dce-8ca5-203b4f79dfb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:41.566060 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757eaf0-0b37-4dce-8ca5-203b4f79dfb2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:41.586008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-3757eaf0-0b37-4dce-8ca5-203b4f79dfb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:25:41.586008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:25:41.586008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:25:41.586008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:25:41.586008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:25:41.586008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:25:41.586008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:25:41.586008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:25:41.586008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:25:41.586008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:25:41.586008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:25:41.586008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:25:41.586008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:25:41.586008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:25:41.586008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:25:41.586008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:25:41.587027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:25:41.587959 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:25:41.587959 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:25:41.587959 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:25:41.587959 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 9/18] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:25:41.592782 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757eaf0-0b37-4dce-8ca5-203b4f79dfb2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:41.614298 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:41.653197 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0b908-b296-48e6-b901-0f25e55cb683 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:41.689089 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622a25b-2408-4753-bab2-b9fe35c55224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:41.689717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622a25b-2408-4753-bab2-b9fe35c55224 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:41.689991 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622a25b-2408-4753-bab2-b9fe35c55224 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:41.690262 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622a25b-2408-4753-bab2-b9fe35c55224 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:41.691405 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622a25b-2408-4753-bab2-b9fe35c55224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:41.692667 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757eaf0-0b37-4dce-8ca5-203b4f79dfb2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:41.693131 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622a25b-2408-4753-bab2-b9fe35c55224 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:41.694370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 10/19] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:25:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:25:41.699982 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622a25b-2408-4753-bab2-b9fe35c55224 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:41.729432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3c16e-e52d-43e5-9203-722d5b64a58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:41.730027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3c16e-e52d-43e5-9203-722d5b64a58b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:41.730295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3c16e-e52d-43e5-9203-722d5b64a58b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:41.730556 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3c16e-e52d-43e5-9203-722d5b64a58b None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:41.731217 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3c16e-e52d-43e5-9203-722d5b64a58b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:25:41.731507 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3c16e-e52d-43e5-9203-722d5b64a58b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:41.732865 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3c16e-e52d-43e5-9203-722d5b64a58b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:41.750113 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 10/20] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:25:41.783320 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:41.804785 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622a25b-2408-4753-bab2-b9fe35c55224 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:41.841638 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa439041-9545-4561-815d-e9a7f78e76d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:41.842312 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa439041-9545-4561-815d-e9a7f78e76d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:41.842586 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa439041-9545-4561-815d-e9a7f78e76d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:41.842856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa439041-9545-4561-815d-e9a7f78e76d9 None admin] PATH_INFO: `/v3/domains/default/users/3ce73a2078a24b19af860a9906b4d83e/roles/4b1f48ab911e4fadbc70164ae43d508b` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:41.845406 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa439041-9545-4561-815d-e9a7f78e76d9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3ce73a2078a24b19af860a9906b4d83e, role_id=4b1f48ab911e4fadbc70164ae43d508b)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:41.846870 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa439041-9545-4561-815d-e9a7f78e76d9 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:41.962300 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 11/21] 10.222.0.16 () {58 vars in 1361 bytes} [Wed Oct 12 14:25:41 2022] PUT /identity/v3/domains/default/users/3ce73a2078a24b19af860a9906b4d83e/roles/4b1f48ab911e4fadbc70164ae43d508b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 14:25:44.194881 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164a03a-3705-4770-aee0-d165fceea68d None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:44.195428 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164a03a-3705-4770-aee0-d165fceea68d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:44.195501 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164a03a-3705-4770-aee0-d165fceea68d None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:44.197121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 11/22] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:25:44.204062 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f44efa-473c-45bb-a6bf-10730fa15d55 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:44.204510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f44efa-473c-45bb-a6bf-10730fa15d55 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:44.204690 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f44efa-473c-45bb-a6bf-10730fa15d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:44.232297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-d7f44efa-473c-45bb-a6bf-10730fa15d55 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:44.233896 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-d7f44efa-473c-45bb-a6bf-10730fa15d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:44.321224 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 12/23] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:25:44.722605 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7963a5-67e6-4b6b-8100-f20645506584 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:44.722968 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7963a5-67e6-4b6b-8100-f20645506584 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:44.723210 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7963a5-67e6-4b6b-8100-f20645506584 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:44.724976 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 12/24] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:25:44.730514 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9c549-676f-4b97-b724-6de992eb5c46 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:44.730796 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9c549-676f-4b97-b724-6de992eb5c46 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:44.731066 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9c549-676f-4b97-b724-6de992eb5c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:44.756384 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-bbb9c549-676f-4b97-b724-6de992eb5c46 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:44.757602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-bbb9c549-676f-4b97-b724-6de992eb5c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:44.837242 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:44.848174 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 13/25] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:25:44.865607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87638b8-3b1e-4e33-94a5-477d722b9d07 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:44.865899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87638b8-3b1e-4e33-94a5-477d722b9d07 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:44.866147 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87638b8-3b1e-4e33-94a5-477d722b9d07 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:44.867834 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 13/26] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:25:44.869265 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23f1a8-9101-4d6e-a65d-b0aef63db93f None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:44.869546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23f1a8-9101-4d6e-a65d-b0aef63db93f None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:44.869814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23f1a8-9101-4d6e-a65d-b0aef63db93f None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:44.874550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 14/27] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:25:44.879639 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01094f-ee1e-4a9f-99f5-baaf22558e31 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:44.879922 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01094f-ee1e-4a9f-99f5-baaf22558e31 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:44.880175 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01094f-ee1e-4a9f-99f5-baaf22558e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:44.886020 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84803761-e836-4bb3-a5ae-14e6290a7258 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:44.886298 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84803761-e836-4bb3-a5ae-14e6290a7258 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:44.886565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84803761-e836-4bb3-a5ae-14e6290a7258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:44.914031 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9f01094f-ee1e-4a9f-99f5-baaf22558e31 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:44.915242 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9f01094f-ee1e-4a9f-99f5-baaf22558e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:44.931285 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-84803761-e836-4bb3-a5ae-14e6290a7258 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:44.932437 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-84803761-e836-4bb3-a5ae-14e6290a7258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:45.008742 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 14/28] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:25:45.009604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01094f-ee1e-4a9f-99f5-baaf22558e31 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:45.030732 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:45.052519 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 15/29] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:25:45.131003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01094f-ee1e-4a9f-99f5-baaf22558e31 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:45.165881 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0773502-da7f-4b8f-bc8c-5c62a7a36c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:45.166519 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0773502-da7f-4b8f-bc8c-5c62a7a36c40 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:45.166794 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0773502-da7f-4b8f-bc8c-5c62a7a36c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:45.167064 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0773502-da7f-4b8f-bc8c-5c62a7a36c40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:45.177800 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0773502-da7f-4b8f-bc8c-5c62a7a36c40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:45.179327 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0773502-da7f-4b8f-bc8c-5c62a7a36c40 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:45.195209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-e0773502-da7f-4b8f-bc8c-5c62a7a36c40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:25:45.195209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:25:45.195209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:25:45.195209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:25:45.195209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:25:45.195209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:25:45.195209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:25:45.195209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:25:45.195209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:25:45.195209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:25:45.195209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:25:45.195209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:25:45.195209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:25:45.195209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:25:45.195209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:25:45.195209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:25:45.197347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:25:45.199312 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:25:45.199312 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:25:45.199312 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:25:45.199312 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 15/30] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:25:45.204478 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84803761-e836-4bb3-a5ae-14e6290a7258 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:45.251032 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0773502-da7f-4b8f-bc8c-5c62a7a36c40 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:45.375114 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:45.415442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84803761-e836-4bb3-a5ae-14e6290a7258 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:45.448286 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:45.456548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0773502-da7f-4b8f-bc8c-5c62a7a36c40 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:45.457429 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840652a-1fe2-4857-b34f-407acc00304a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:45.458512 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840652a-1fe2-4857-b34f-407acc00304a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:45.458982 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840652a-1fe2-4857-b34f-407acc00304a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:45.459509 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840652a-1fe2-4857-b34f-407acc00304a None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:45.460356 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840652a-1fe2-4857-b34f-407acc00304a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:25:45.460866 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840652a-1fe2-4857-b34f-407acc00304a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:45.462650 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840652a-1fe2-4857-b34f-407acc00304a None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:45.485946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 16/31] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:45.487230 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840652a-1fe2-4857-b34f-407acc00304a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:45.543279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c732d0c-be9c-4c55-b454-8d6f3b4398be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:45.543901 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c732d0c-be9c-4c55-b454-8d6f3b4398be None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:45.544174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c732d0c-be9c-4c55-b454-8d6f3b4398be None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:45.544440 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c732d0c-be9c-4c55-b454-8d6f3b4398be None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:45.545130 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c732d0c-be9c-4c55-b454-8d6f3b4398be None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:45.547618 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c732d0c-be9c-4c55-b454-8d6f3b4398be None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:45.631289 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840652a-1fe2-4857-b34f-407acc00304a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:45.667952 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a71f20-af1b-497e-8180-000939eb6690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:45.668505 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a71f20-af1b-497e-8180-000939eb6690 None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:45.668782 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a71f20-af1b-497e-8180-000939eb6690 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:45.669051 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a71f20-af1b-497e-8180-000939eb6690 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:45.669725 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a71f20-af1b-497e-8180-000939eb6690 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:45.671680 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a71f20-af1b-497e-8180-000939eb6690 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:45.690891 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 17/32] 10.222.0.16 () {60 vars in 1131 bytes} [Wed Oct 12 14:25:45 2022] POST /identity/v3/roles => generated 219 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:25:45.691534 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a71f20-af1b-497e-8180-000939eb6690 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:45.704252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 16/33] 10.222.0.16 () {60 vars in 1131 bytes} [Wed Oct 12 14:25:45 2022] POST /identity/v3/roles => generated 225 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 14:25:45.705708 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c732d0c-be9c-4c55-b454-8d6f3b4398be None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:45.755922 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:45.819971 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a71f20-af1b-497e-8180-000939eb6690 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:45.862073 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882ab37-79e0-438c-98bf-b224de751c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:45.863234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882ab37-79e0-438c-98bf-b224de751c87 None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:45.863995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882ab37-79e0-438c-98bf-b224de751c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:45.864512 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882ab37-79e0-438c-98bf-b224de751c87 None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:45.865475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882ab37-79e0-438c-98bf-b224de751c87 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:45.867239 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882ab37-79e0-438c-98bf-b224de751c87 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:45.928644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 18/34] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Oct 12 14:25:45 2022] POST /identity/v3/projects => generated 301 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:25:45.933042 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a22a0-56d8-47c2-b602-1323f9ae871f None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:45.934473 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a22a0-56d8-47c2-b602-1323f9ae871f None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:45.935328 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a22a0-56d8-47c2-b602-1323f9ae871f None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:45.939114 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 19/35] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:25:45.941467 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c732d0c-be9c-4c55-b454-8d6f3b4398be None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:45.944391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f16e41-456b-4ac9-8ac5-8fa7af738afd None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:45.944953 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f16e41-456b-4ac9-8ac5-8fa7af738afd None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:45.945422 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f16e41-456b-4ac9-8ac5-8fa7af738afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:45.985559 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-c1f16e41-456b-4ac9-8ac5-8fa7af738afd None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:45.992178 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-c1f16e41-456b-4ac9-8ac5-8fa7af738afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:46.003585 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:46.032946 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1422f-5f13-446c-841f-00ecf803d572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:46.034204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1422f-5f13-446c-841f-00ecf803d572 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:46.036096 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1422f-5f13-446c-841f-00ecf803d572 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:46.036610 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1422f-5f13-446c-841f-00ecf803d572 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:46.059533 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c1422f-5f13-446c-841f-00ecf803d572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:25:46.068451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-b3c1422f-5f13-446c-841f-00ecf803d572 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:25:46.068451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:25:46.068451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:25:46.068451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:25:46.068451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:25:46.068451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:25:46.068451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:25:46.068451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:25:46.068451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:25:46.068451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:25:46.068451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:25:46.068451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:25:46.068451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:25:46.068451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:25:46.068451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:25:46.068451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:25:46.069376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:25:46.069376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:25:46.069376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:25:46.069376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:25:46.069376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:25:46.069376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:25:46.069376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:25:46.069376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:25:46.069376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:25:46.069376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:25:46.069376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:25:46.069376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:25:46.069376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:25:46.069376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:25:46.069376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:25:46.069376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:25:46.070100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:25:46.070100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:25:46.070100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:25:46.070100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:25:46.070100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:25:46.070100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:25:46.070100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:25:46.070100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:25:46.070100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:25:46.070100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:25:46.070100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:25:46.070100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:25:46.070100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 17/36] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:25:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:25:46.078041 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1422f-5f13-446c-841f-00ecf803d572 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:46.114461 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 20/37] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:25:46.116141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddb50e-c98c-4040-84b1-73b928d5a339 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:46.116673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddb50e-c98c-4040-84b1-73b928d5a339 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:46.117172 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddb50e-c98c-4040-84b1-73b928d5a339 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:46.118844 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 21/38] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:25:46.125177 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa169825-b3d5-4fa2-bda6-38cdea069e5c None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:46.125455 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa169825-b3d5-4fa2-bda6-38cdea069e5c None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:46.125729 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa169825-b3d5-4fa2-bda6-38cdea069e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:46.147397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-fa169825-b3d5-4fa2-bda6-38cdea069e5c None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:46.149033 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-fa169825-b3d5-4fa2-bda6-38cdea069e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:46.184039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:46.212240 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1422f-5f13-446c-841f-00ecf803d572 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:46.244332 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 22/39] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:25:46.258636 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecc6f6-f27d-4fbe-935b-ec66775a76e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:46.259293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecc6f6-f27d-4fbe-935b-ec66775a76e3 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:46.259556 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecc6f6-f27d-4fbe-935b-ec66775a76e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:46.259832 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecc6f6-f27d-4fbe-935b-ec66775a76e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:46.260498 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecc6f6-f27d-4fbe-935b-ec66775a76e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:25:46.260795 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecc6f6-f27d-4fbe-935b-ec66775a76e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:46.262302 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecc6f6-f27d-4fbe-935b-ec66775a76e3 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:46.275736 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 18/40] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:25:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:25:46.282836 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa169825-b3d5-4fa2-bda6-38cdea069e5c None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:46.437365 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecc6f6-f27d-4fbe-935b-ec66775a76e3 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:46.492171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:46.496891 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa169825-b3d5-4fa2-bda6-38cdea069e5c None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:46.535471 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9e984-2b5c-47fa-801b-0e07f9c1fbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:46.536106 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9e984-2b5c-47fa-801b-0e07f9c1fbdd None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:46.536385 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9e984-2b5c-47fa-801b-0e07f9c1fbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:46.536659 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9e984-2b5c-47fa-801b-0e07f9c1fbdd None admin] PATH_INFO: `/v3/users/3ce73a2078a24b19af860a9906b4d83e` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:46.537773 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9e984-2b5c-47fa-801b-0e07f9c1fbdd None admin] RBAC: Authorizing `identity:get_user(user_id=3ce73a2078a24b19af860a9906b4d83e)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:46.539260 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9e984-2b5c-47fa-801b-0e07f9c1fbdd None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:46.540854 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 23/41] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:25:46 2022] GET /identity/v3/users/3ce73a2078a24b19af860a9906b4d83e => generated 247 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:46.541604 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9e984-2b5c-47fa-801b-0e07f9c1fbdd None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:46.551849 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecc6f6-f27d-4fbe-935b-ec66775a76e3 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:46.576034 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:46.588188 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ec90a-4879-45bc-acbc-ce7c21fbc44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:46.588921 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ec90a-4879-45bc-acbc-ce7c21fbc44b None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:46.589205 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ec90a-4879-45bc-acbc-ce7c21fbc44b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:46.589480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ec90a-4879-45bc-acbc-ce7c21fbc44b None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:46.590160 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ec90a-4879-45bc-acbc-ce7c21fbc44b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:46.591584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ec90a-4879-45bc-acbc-ce7c21fbc44b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:46.605854 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 19/42] 10.222.0.16 () {60 vars in 1131 bytes} [Wed Oct 12 14:25:46 2022] POST /identity/v3/roles => generated 223 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 14:25:46.606648 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ec90a-4879-45bc-acbc-ce7c21fbc44b None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:46.665537 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9e984-2b5c-47fa-801b-0e07f9c1fbdd None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:46.706951 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66a190-13b2-4627-88b3-d4783892bcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:46.707850 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66a190-13b2-4627-88b3-d4783892bcae None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:46.708190 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66a190-13b2-4627-88b3-d4783892bcae None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:46.708523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66a190-13b2-4627-88b3-d4783892bcae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:46.710122 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66a190-13b2-4627-88b3-d4783892bcae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:46.711735 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66a190-13b2-4627-88b3-d4783892bcae None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:46.713845 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 24/43] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:25:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:46.721356 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66a190-13b2-4627-88b3-d4783892bcae None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:46.728686 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ec90a-4879-45bc-acbc-ce7c21fbc44b None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:46.732880 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:46.783018 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a94706-1146-4f9b-8f41-cefb6634d117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:46.783686 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a94706-1146-4f9b-8f41-cefb6634d117 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:46.783973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a94706-1146-4f9b-8f41-cefb6634d117 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:46.784222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a94706-1146-4f9b-8f41-cefb6634d117 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:46.785421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a94706-1146-4f9b-8f41-cefb6634d117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:46.786847 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a94706-1146-4f9b-8f41-cefb6634d117 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:46.788428 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 20/44] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:25:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:46.794942 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a94706-1146-4f9b-8f41-cefb6634d117 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:46.835627 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:46.839576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66a190-13b2-4627-88b3-d4783892bcae None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:46.874114 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07917e8-173b-40f4-aaab-c627562ef6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:46.874920 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07917e8-173b-40f4-aaab-c627562ef6ac None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:46.875265 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07917e8-173b-40f4-aaab-c627562ef6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:46.875544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07917e8-173b-40f4-aaab-c627562ef6ac None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:46.876201 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07917e8-173b-40f4-aaab-c627562ef6ac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:46.877652 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07917e8-173b-40f4-aaab-c627562ef6ac None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:46.899074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a94706-1146-4f9b-8f41-cefb6634d117 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:46.903771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 25/45] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Oct 12 14:25:46 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 14:25:46.942292 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e65a4-16f7-4b1b-a2b4-ced254121c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:46.942923 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e65a4-16f7-4b1b-a2b4-ced254121c74 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:46.943231 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e65a4-16f7-4b1b-a2b4-ced254121c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:46.943506 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e65a4-16f7-4b1b-a2b4-ced254121c74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:46.944256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e65a4-16f7-4b1b-a2b4-ced254121c74 None admin] RBAC: Adding query filter params (role.id=4b1f48ab911e4fadbc70164ae43d508b, scope.domain.id=default, user.id=3ce73a2078a24b19af860a9906b4d83e) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:25:46.944559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e65a4-16f7-4b1b-a2b4-ced254121c74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:46.946017 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e65a4-16f7-4b1b-a2b4-ced254121c74 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:46.955699 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 21/46] 10.222.0.16 () {56 vars in 1322 bytes} [Wed Oct 12 14:25:46 2022] GET /identity/v3/role_assignments?user.id=3ce73a2078a24b19af860a9906b4d83e&scope.domain.id=default&role.id=4b1f48ab911e4fadbc70164ae43d508b => generated 530 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:25:49.051177 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d8dca-38d4-4b7b-b933-3bc36e1e1f50 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:49.051697 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d8dca-38d4-4b7b-b933-3bc36e1e1f50 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:49.051778 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d8dca-38d4-4b7b-b933-3bc36e1e1f50 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:49.053080 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 26/47] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:25:49.058225 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f948b-ccf0-4757-b43e-63a0e311690d None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:49.058633 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f948b-ccf0-4757-b43e-63a0e311690d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:49.058783 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f948b-ccf0-4757-b43e-63a0e311690d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:49.078124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-618f948b-ccf0-4757-b43e-63a0e311690d None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:49.079636 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-618f948b-ccf0-4757-b43e-63a0e311690d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:49.107197 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:49.144996 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 22/48] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:25:49.412086 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d8dca-38d4-4b7b-b933-3bc36e1e1f50 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:49.509565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d8dca-38d4-4b7b-b933-3bc36e1e1f50 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:49.542598 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87345d9b-3d1a-4695-97bd-46b699d05b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:49.543234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87345d9b-3d1a-4695-97bd-46b699d05b02 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:49.543511 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87345d9b-3d1a-4695-97bd-46b699d05b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:49.543780 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87345d9b-3d1a-4695-97bd-46b699d05b02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:49.544886 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87345d9b-3d1a-4695-97bd-46b699d05b02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:49.547119 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87345d9b-3d1a-4695-97bd-46b699d05b02 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:49.548680 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 27/49] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:25:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:49.554099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f948b-ccf0-4757-b43e-63a0e311690d None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:49.649018 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f948b-ccf0-4757-b43e-63a0e311690d None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:49.682248 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01aa9b-fa5c-4bfd-a4d3-739098257aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:49.682790 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01aa9b-fa5c-4bfd-a4d3-739098257aa9 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:49.683063 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01aa9b-fa5c-4bfd-a4d3-739098257aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:49.683363 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01aa9b-fa5c-4bfd-a4d3-739098257aa9 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:49.684012 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01aa9b-fa5c-4bfd-a4d3-739098257aa9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:49.685449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01aa9b-fa5c-4bfd-a4d3-739098257aa9 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:49.711016 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 23/50] 10.222.0.16 () {60 vars in 1140 bytes} [Wed Oct 12 14:25:49 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:25:51.845503 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53b800-eca3-4773-a9b5-7ddafdb75bc4 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:51.845987 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53b800-eca3-4773-a9b5-7ddafdb75bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:51.845987 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53b800-eca3-4773-a9b5-7ddafdb75bc4 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:51.847468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 28/51] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:25:51.852483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4196a-51c9-4681-bfec-186db440d66f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:51.852844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4196a-51c9-4681-bfec-186db440d66f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:51.853030 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4196a-51c9-4681-bfec-186db440d66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:51.871550 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a9d4196a-51c9-4681-bfec-186db440d66f None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:51.872747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a9d4196a-51c9-4681-bfec-186db440d66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:51.885386 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:51.937875 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 24/52] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:25:52.202401 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53b800-eca3-4773-a9b5-7ddafdb75bc4 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:52.221534 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:52.302172 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53b800-eca3-4773-a9b5-7ddafdb75bc4 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:52.334662 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2f7d2-480c-46d9-ae50-50ceb296a06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:52.335298 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2f7d2-480c-46d9-ae50-50ceb296a06e None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:52.335575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2f7d2-480c-46d9-ae50-50ceb296a06e None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:52.335843 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2f7d2-480c-46d9-ae50-50ceb296a06e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:52.336936 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2f7d2-480c-46d9-ae50-50ceb296a06e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:52.338375 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2f7d2-480c-46d9-ae50-50ceb296a06e None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:52.339935 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 29/53] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:25:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:52.345693 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4196a-51c9-4681-bfec-186db440d66f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:52.443530 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4196a-51c9-4681-bfec-186db440d66f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:52.476556 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1f2a4-597c-485e-8512-c2754c88aef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:52.477094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1f2a4-597c-485e-8512-c2754c88aef9 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:52.477371 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1f2a4-597c-485e-8512-c2754c88aef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:52.477641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1f2a4-597c-485e-8512-c2754c88aef9 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:52.478285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1f2a4-597c-485e-8512-c2754c88aef9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:52.479758 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1f2a4-597c-485e-8512-c2754c88aef9 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:52.506532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 25/54] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:25:52 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:25:55.932844 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8430ddd-0c86-4daf-adf0-6cc88e8e9b8a None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:55.933411 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8430ddd-0c86-4daf-adf0-6cc88e8e9b8a None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:55.933411 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8430ddd-0c86-4daf-adf0-6cc88e8e9b8a None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:55.934921 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 30/55] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:25:55.962736 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079ee73-ca14-47bb-9770-e54e591e034a None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:55.963275 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079ee73-ca14-47bb-9770-e54e591e034a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:55.963275 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079ee73-ca14-47bb-9770-e54e591e034a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:55.975790 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:55.994290 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3079ee73-ca14-47bb-9770-e54e591e034a None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:55.995632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3079ee73-ca14-47bb-9770-e54e591e034a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:56.092259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4336dda-65e2-4958-aee6-ab3fb2d214d3 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:56.092614 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4336dda-65e2-4958-aee6-ab3fb2d214d3 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:56.092797 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4336dda-65e2-4958-aee6-ab3fb2d214d3 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:56.094146 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 31/56] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:25:56.100305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3436ca-6ac4-4994-a5f9-74245106f362 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:56.100580 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3436ca-6ac4-4994-a5f9-74245106f362 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:56.100849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3436ca-6ac4-4994-a5f9-74245106f362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:56.132900 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-ef3436ca-6ac4-4994-a5f9-74245106f362 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:56.134316 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-ef3436ca-6ac4-4994-a5f9-74245106f362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:56.211783 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 32/57] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:25:56.211942 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 26/58] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:25:56.212981 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7a06a-c6a1-4a57-82fc-13ea38938879 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:56.213435 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7a06a-c6a1-4a57-82fc-13ea38938879 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:56.213851 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7a06a-c6a1-4a57-82fc-13ea38938879 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:56.215482 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 27/59] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:25:56.224521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faf7c2-d0b5-4e78-a858-4b85f54a06e4 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:56.224798 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faf7c2-d0b5-4e78-a858-4b85f54a06e4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:56.225073 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faf7c2-d0b5-4e78-a858-4b85f54a06e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:56.260163 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-53faf7c2-d0b5-4e78-a858-4b85f54a06e4 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:56.261350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-53faf7c2-d0b5-4e78-a858-4b85f54a06e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:56.366317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 28/60] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:25:56.434081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceef0eb-8556-4243-ba19-2b98dc2909ab None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:56.434081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceef0eb-8556-4243-ba19-2b98dc2909ab None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:56.434081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceef0eb-8556-4243-ba19-2b98dc2909ab None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:56.434081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 33/61] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:25:56.440734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b6047-4f12-452c-9517-159e8c0a098e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:56.441006 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b6047-4f12-452c-9517-159e8c0a098e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:56.441281 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b6047-4f12-452c-9517-159e8c0a098e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:56.468191 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-775b6047-4f12-452c-9517-159e8c0a098e None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:56.469295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-775b6047-4f12-452c-9517-159e8c0a098e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:56.541164 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b42f8-3d95-4a8b-8ed7-c585b2e0443a None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:56.541472 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b42f8-3d95-4a8b-8ed7-c585b2e0443a None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:56.541704 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b42f8-3d95-4a8b-8ed7-c585b2e0443a None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:56.543218 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 34/62] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:25:56.545187 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b42f8-3d95-4a8b-8ed7-c585b2e0443a None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:56.545297 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:56.556000 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 29/63] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:25:56.557468 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5a5fc-cff2-4f34-8b7a-7db8b9ecab82 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:56.557748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5a5fc-cff2-4f34-8b7a-7db8b9ecab82 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:56.558018 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5a5fc-cff2-4f34-8b7a-7db8b9ecab82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:56.569573 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:56.582323 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-21f5a5fc-cff2-4f34-8b7a-7db8b9ecab82 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:25:56.584209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-21f5a5fc-cff2-4f34-8b7a-7db8b9ecab82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:25:56.671089 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 30/64] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:25:56.672034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5a5fc-cff2-4f34-8b7a-7db8b9ecab82 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:56.680791 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b42f8-3d95-4a8b-8ed7-c585b2e0443a None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:56.725071 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370c9de-7c52-47c2-9e56-4446280bbc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:56.725913 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370c9de-7c52-47c2-9e56-4446280bbc9a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:56.726429 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370c9de-7c52-47c2-9e56-4446280bbc9a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:56.726843 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370c9de-7c52-47c2-9e56-4446280bbc9a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:56.738816 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370c9de-7c52-47c2-9e56-4446280bbc9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:25:56.744248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-e370c9de-7c52-47c2-9e56-4446280bbc9a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:25:56.744248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:25:56.744248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:25:56.744248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:25:56.744248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:25:56.744248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:25:56.744248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:25:56.744248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:25:56.744248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:25:56.744248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:25:56.744248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:25:56.744248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:25:56.744248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:25:56.744248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:25:56.744248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:25:56.744248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:25:56.745270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:25:56.745270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:25:56.745270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:25:56.745270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:25:56.745270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:25:56.745270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:25:56.745270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:25:56.745270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:25:56.745270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:25:56.745270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:25:56.745270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:25:56.745270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:25:56.745270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:25:56.745270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:25:56.745270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:25:56.745270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:25:56.746078 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:25:56.746078 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:25:56.746078 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:25:56.746078 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:25:56.746078 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:25:56.746078 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:25:56.746078 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:25:56.746078 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:25:56.746078 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:25:56.746078 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:25:56.746078 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:25:56.746078 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:25:56.748261 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 35/65] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Oct 12 14:25:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:25:56.749226 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370c9de-7c52-47c2-9e56-4446280bbc9a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:56.789295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5a5fc-cff2-4f34-8b7a-7db8b9ecab82 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:56.817030 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:56.830635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b48353-5ab5-476b-80e0-b5ab6163f599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:56.831490 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b48353-5ab5-476b-80e0-b5ab6163f599 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:56.831903 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b48353-5ab5-476b-80e0-b5ab6163f599 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:56.832319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b48353-5ab5-476b-80e0-b5ab6163f599 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:56.844068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b48353-5ab5-476b-80e0-b5ab6163f599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:25:56.846482 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:56.849419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-e4b48353-5ab5-476b-80e0-b5ab6163f599 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:25:56.849419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:25:56.849419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:25:56.849419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:25:56.849419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:25:56.849419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:25:56.849419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:25:56.849419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:25:56.849419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:25:56.849419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:25:56.849419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:25:56.849419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:25:56.849419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:25:56.849419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:25:56.849419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:25:56.849419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:25:56.850253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:25:56.850253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:25:56.850253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:25:56.850253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:25:56.850253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:25:56.850253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:25:56.850253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:25:56.850253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:25:56.850253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:25:56.850253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:25:56.850253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:25:56.850253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:25:56.850253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:25:56.850253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:25:56.850253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:25:56.850253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:25:56.850979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:25:56.850979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:25:56.850979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:25:56.850979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:25:56.850979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:25:56.850979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:25:56.850979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:25:56.850979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:25:56.850979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:25:56.850979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:25:56.850979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:25:56.850979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:25:56.850979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 31/66] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:25:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:25:56.852085 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b48353-5ab5-476b-80e0-b5ab6163f599 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:56.872865 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370c9de-7c52-47c2-9e56-4446280bbc9a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:56.907379 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07caa43d-9830-42b0-97da-f543d9e3fd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:56.908014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07caa43d-9830-42b0-97da-f543d9e3fd4a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:56.908289 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07caa43d-9830-42b0-97da-f543d9e3fd4a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:56.908562 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07caa43d-9830-42b0-97da-f543d9e3fd4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:56.918507 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07caa43d-9830-42b0-97da-f543d9e3fd4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:25:56.924188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-07caa43d-9830-42b0-97da-f543d9e3fd4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:25:56.924188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:25:56.924188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:25:56.924188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:25:56.924188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:25:56.924188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:25:56.924188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:25:56.924188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:25:56.924188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:25:56.924188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:25:56.924188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:25:56.924188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:25:56.924188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:25:56.924188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:25:56.924188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:25:56.924188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:25:56.925003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:25:56.925003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:25:56.925003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:25:56.925003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:25:56.925003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:25:56.925003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:25:56.925003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:25:56.925003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:25:56.925003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:25:56.925003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:25:56.925003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:25:56.925003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:25:56.925003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:25:56.925003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:25:56.925003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:25:56.925003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:25:56.925717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:25:56.925717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:25:56.925717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:25:56.925717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:25:56.925717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:25:56.925717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:25:56.925717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:25:56.925717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:25:56.925717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:25:56.925717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:25:56.925717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:25:56.925717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:25:56.925717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 36/67] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:25:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:25:56.926312 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07caa43d-9830-42b0-97da-f543d9e3fd4a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:56.972755 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b48353-5ab5-476b-80e0-b5ab6163f599 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:57.006631 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab68b0-4b7b-4e39-8b25-91cc4c826148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:57.007189 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab68b0-4b7b-4e39-8b25-91cc4c826148 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:57.007468 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab68b0-4b7b-4e39-8b25-91cc4c826148 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:57.007744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab68b0-4b7b-4e39-8b25-91cc4c826148 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:57.008391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab68b0-4b7b-4e39-8b25-91cc4c826148 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:25:57.008687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab68b0-4b7b-4e39-8b25-91cc4c826148 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:57.010067 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab68b0-4b7b-4e39-8b25-91cc4c826148 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:57.018048 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 32/68] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:25:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:57.018827 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab68b0-4b7b-4e39-8b25-91cc4c826148 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:57.025750 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07caa43d-9830-42b0-97da-f543d9e3fd4a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:57.060325 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d761c-96cf-44eb-86e0-bc4641f973f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:57.060952 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d761c-96cf-44eb-86e0-bc4641f973f0 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:57.061231 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d761c-96cf-44eb-86e0-bc4641f973f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:57.061500 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d761c-96cf-44eb-86e0-bc4641f973f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:57.062651 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d761c-96cf-44eb-86e0-bc4641f973f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:57.064158 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d761c-96cf-44eb-86e0-bc4641f973f0 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:57.065709 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 37/69] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:25:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:57.066465 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d761c-96cf-44eb-86e0-bc4641f973f0 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:57.069051 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:57.071965 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:57.126077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab68b0-4b7b-4e39-8b25-91cc4c826148 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:57.159334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b0fd8-4223-428a-951f-fccf43ec8ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:57.159937 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b0fd8-4223-428a-951f-fccf43ec8ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:57.160213 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b0fd8-4223-428a-951f-fccf43ec8ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:57.160483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b0fd8-4223-428a-951f-fccf43ec8ab7 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:57.161136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b0fd8-4223-428a-951f-fccf43ec8ab7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:25:57.161435 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b0fd8-4223-428a-951f-fccf43ec8ab7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:57.162801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b0fd8-4223-428a-951f-fccf43ec8ab7 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:57.168088 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d761c-96cf-44eb-86e0-bc4641f973f0 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:57.170840 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 33/70] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:25:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:57.171653 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b0fd8-4223-428a-951f-fccf43ec8ab7 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:57.201947 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4554d0f-f0b2-4231-bf99-8cb59a51b866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:57.202550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4554d0f-f0b2-4231-bf99-8cb59a51b866 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:57.202854 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4554d0f-f0b2-4231-bf99-8cb59a51b866 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:57.203120 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4554d0f-f0b2-4231-bf99-8cb59a51b866 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:57.203811 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4554d0f-f0b2-4231-bf99-8cb59a51b866 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:25:57.204110 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4554d0f-f0b2-4231-bf99-8cb59a51b866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:57.205472 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4554d0f-f0b2-4231-bf99-8cb59a51b866 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:57.213439 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 38/71] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:25:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:57.214227 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4554d0f-f0b2-4231-bf99-8cb59a51b866 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:57.272980 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b0fd8-4223-428a-951f-fccf43ec8ab7 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:57.279579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:57.282169 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:57.307552 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e28fd-213a-4703-9054-bb131393854a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:57.308174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e28fd-213a-4703-9054-bb131393854a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:57.308447 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e28fd-213a-4703-9054-bb131393854a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:57.308718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e28fd-213a-4703-9054-bb131393854a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:57.313978 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4554d0f-f0b2-4231-bf99-8cb59a51b866 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:57.319365 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e28fd-213a-4703-9054-bb131393854a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:25:57.324737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-385e28fd-213a-4703-9054-bb131393854a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:25:57.324737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:25:57.324737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:25:57.324737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:25:57.324737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:25:57.324737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:25:57.324737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:25:57.324737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:25:57.324737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:25:57.324737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:25:57.324737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:25:57.324737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:25:57.324737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:25:57.324737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:25:57.324737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:25:57.324737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:25:57.325645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:25:57.325645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:25:57.325645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:25:57.325645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:25:57.325645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:25:57.325645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:25:57.325645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:25:57.325645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:25:57.325645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:25:57.325645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:25:57.325645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:25:57.325645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:25:57.325645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:25:57.325645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:25:57.325645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:25:57.325645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:25:57.326385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:25:57.326385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:25:57.326385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:25:57.326385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:25:57.326385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:25:57.326385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:25:57.326385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:25:57.326385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:25:57.326385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:25:57.326385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:25:57.326385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:25:57.326385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:25:57.326385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 34/72] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:25:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:25:57.327006 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e28fd-213a-4703-9054-bb131393854a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:57.348749 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac45902-c856-4f61-a037-517c26844468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:57.349374 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac45902-c856-4f61-a037-517c26844468 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:57.349650 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac45902-c856-4f61-a037-517c26844468 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:57.349921 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac45902-c856-4f61-a037-517c26844468 None admin] PATH_INFO: `/v3/users/0b1931fa42bd48ffae39f00dd53304fb` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:57.363727 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac45902-c856-4f61-a037-517c26844468 None admin] RBAC: Authorizing `identity:get_user(user_id=0b1931fa42bd48ffae39f00dd53304fb)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:57.365172 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac45902-c856-4f61-a037-517c26844468 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:57.366759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 39/73] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:25:57 2022] GET /identity/v3/users/0b1931fa42bd48ffae39f00dd53304fb => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:57.372055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac45902-c856-4f61-a037-517c26844468 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:57.424902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e28fd-213a-4703-9054-bb131393854a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:57.457839 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2c3fc-283b-4322-a874-0374004a2873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:57.458439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2c3fc-283b-4322-a874-0374004a2873 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:57.458716 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2c3fc-283b-4322-a874-0374004a2873 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:57.458989 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2c3fc-283b-4322-a874-0374004a2873 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:57.459665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2c3fc-283b-4322-a874-0374004a2873 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:57.461100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2c3fc-283b-4322-a874-0374004a2873 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:57.468226 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac45902-c856-4f61-a037-517c26844468 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:57.483817 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 35/74] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:25:57 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:25:57.484638 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2c3fc-283b-4322-a874-0374004a2873 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:57.496676 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:57.508852 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e485d9-81b5-428a-9030-c41f7ef3d285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:57.509918 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e485d9-81b5-428a-9030-c41f7ef3d285 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:57.510197 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e485d9-81b5-428a-9030-c41f7ef3d285 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:57.510465 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e485d9-81b5-428a-9030-c41f7ef3d285 None admin] PATH_INFO: `/v3/users/0b1931fa42bd48ffae39f00dd53304fb` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:57.512105 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e485d9-81b5-428a-9030-c41f7ef3d285 None admin] RBAC: Authorizing `identity:get_user(user_id=0b1931fa42bd48ffae39f00dd53304fb)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:57.513667 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e485d9-81b5-428a-9030-c41f7ef3d285 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:57.515586 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 40/75] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:25:57 2022] GET /identity/v3/users/0b1931fa42bd48ffae39f00dd53304fb => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:57.516193 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e485d9-81b5-428a-9030-c41f7ef3d285 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:57.532575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:57.625669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2c3fc-283b-4322-a874-0374004a2873 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:57.633364 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e485d9-81b5-428a-9030-c41f7ef3d285 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:57.665134 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5516abd-faba-449c-a739-635745a958b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:57.666065 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5516abd-faba-449c-a739-635745a958b8 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:57.666531 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5516abd-faba-449c-a739-635745a958b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:57.666808 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5516abd-faba-449c-a739-635745a958b8 None admin] PATH_INFO: `/v3/users/3ce73a2078a24b19af860a9906b4d83e` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:57.668237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5516abd-faba-449c-a739-635745a958b8 None admin] RBAC: Authorizing `identity:get_user(user_id=3ce73a2078a24b19af860a9906b4d83e)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:57.670224 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5516abd-faba-449c-a739-635745a958b8 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:57.676274 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 36/76] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:25:57 2022] GET /identity/v3/users/3ce73a2078a24b19af860a9906b4d83e => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:57.676274 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5516abd-faba-449c-a739-635745a958b8 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:57.677100 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07293c09-3b59-4837-90a1-f0dbf71f1704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:57.678450 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07293c09-3b59-4837-90a1-f0dbf71f1704 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:57.678450 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07293c09-3b59-4837-90a1-f0dbf71f1704 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:57.678450 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07293c09-3b59-4837-90a1-f0dbf71f1704 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:57.678914 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07293c09-3b59-4837-90a1-f0dbf71f1704 None admin] RBAC: Adding query filter params (name=member) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:25:57.679828 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07293c09-3b59-4837-90a1-f0dbf71f1704 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:57.681227 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07293c09-3b59-4837-90a1-f0dbf71f1704 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:57.690257 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 41/77] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:25:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:57.691173 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07293c09-3b59-4837-90a1-f0dbf71f1704 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:57.752474 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:57.797517 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:57.808781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07293c09-3b59-4837-90a1-f0dbf71f1704 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:57.814037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5516abd-faba-449c-a739-635745a958b8 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:57.854534 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51870bd4-7555-4cb1-aafb-85c850541e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:57.855216 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51870bd4-7555-4cb1-aafb-85c850541e71 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:57.855505 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51870bd4-7555-4cb1-aafb-85c850541e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:57.855780 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51870bd4-7555-4cb1-aafb-85c850541e71 None admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:57.856831 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fb967-8181-40bc-b523-c64e4c8b7813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:57.857131 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51870bd4-7555-4cb1-aafb-85c850541e71 None admin] RBAC: Authorizing `identity:get_project(project_id=ad7b7e0473f64a4685d43fc69df8b643)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:57.857475 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fb967-8181-40bc-b523-c64e4c8b7813 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:57.857750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fb967-8181-40bc-b523-c64e4c8b7813 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:57.858023 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fb967-8181-40bc-b523-c64e4c8b7813 None admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:57.858609 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51870bd4-7555-4cb1-aafb-85c850541e71 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:57.859184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fb967-8181-40bc-b523-c64e4c8b7813 None admin] RBAC: Authorizing `identity:get_project(project_id=ad7b7e0473f64a4685d43fc69df8b643)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:57.860487 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 42/78] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:25:57 2022] GET /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:57.860653 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fb967-8181-40bc-b523-c64e4c8b7813 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:57.861165 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51870bd4-7555-4cb1-aafb-85c850541e71 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:57.862337 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 37/79] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:25:57 2022] GET /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:57.863183 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fb967-8181-40bc-b523-c64e4c8b7813 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:57.979814 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fb967-8181-40bc-b523-c64e4c8b7813 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:57.990674 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51870bd4-7555-4cb1-aafb-85c850541e71 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:58.019656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:58.021756 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9dd8af-1ac8-4732-8055-e45009e66be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:58.022783 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9dd8af-1ac8-4732-8055-e45009e66be6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:58.023614 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9dd8af-1ac8-4732-8055-e45009e66be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:58.024068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9dd8af-1ac8-4732-8055-e45009e66be6 None admin] PATH_INFO: `/v3/users/0b1931fa42bd48ffae39f00dd53304fb` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:58.026356 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9dd8af-1ac8-4732-8055-e45009e66be6 None admin] RBAC: Authorizing `identity:get_user(user_id=0b1931fa42bd48ffae39f00dd53304fb)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:58.028841 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9dd8af-1ac8-4732-8055-e45009e66be6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:58.031720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 38/80] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:25:57 2022] GET /identity/v3/users/0b1931fa42bd48ffae39f00dd53304fb => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:58.032821 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7ade0-d835-4b57-9149-638c1dfd998c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:58.033403 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9dd8af-1ac8-4732-8055-e45009e66be6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:58.033609 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7ade0-d835-4b57-9149-638c1dfd998c None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:58.033970 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7ade0-d835-4b57-9149-638c1dfd998c None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:58.034317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7ade0-d835-4b57-9149-638c1dfd998c None admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:58.035762 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7ade0-d835-4b57-9149-638c1dfd998c None admin] RBAC: Authorizing `identity:get_project(project_id=ad7b7e0473f64a4685d43fc69df8b643)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:58.037361 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7ade0-d835-4b57-9149-638c1dfd998c None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:58.040307 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 43/81] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:25:57 2022] GET /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:58.041216 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7ade0-d835-4b57-9149-638c1dfd998c None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:58.095595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:58.150414 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7ade0-d835-4b57-9149-638c1dfd998c None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:58.155274 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9dd8af-1ac8-4732-8055-e45009e66be6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:58.187512 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8868092-5901-4460-8779-9f4df05079b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:58.188275 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8868092-5901-4460-8779-9f4df05079b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:58.188664 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8868092-5901-4460-8779-9f4df05079b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:58.189012 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8868092-5901-4460-8779-9f4df05079b4 None admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643/users/0b1931fa42bd48ffae39f00dd53304fb/roles/f0761dd38b794ee6bfab00ac01f240cd` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:58.191571 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8868092-5901-4460-8779-9f4df05079b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad7b7e0473f64a4685d43fc69df8b643, user_id=0b1931fa42bd48ffae39f00dd53304fb, role_id=f0761dd38b794ee6bfab00ac01f240cd)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:58.193097 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8868092-5901-4460-8779-9f4df05079b4 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:58.195881 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c60073-c258-45f1-aebc-39427861ab54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:58.196850 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c60073-c258-45f1-aebc-39427861ab54 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:58.197317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c60073-c258-45f1-aebc-39427861ab54 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:58.197863 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c60073-c258-45f1-aebc-39427861ab54 None admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643/users/0b1931fa42bd48ffae39f00dd53304fb/roles/6d1894444bfd400a89591929bd8925de` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:58.200611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c60073-c258-45f1-aebc-39427861ab54 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad7b7e0473f64a4685d43fc69df8b643, user_id=0b1931fa42bd48ffae39f00dd53304fb, role_id=6d1894444bfd400a89591929bd8925de)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:58.202228 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c60073-c258-45f1-aebc-39427861ab54 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:58.208153 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 44/82] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:25:58 2022] PUT /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643/users/0b1931fa42bd48ffae39f00dd53304fb/roles/f0761dd38b794ee6bfab00ac01f240cd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:25:58.209450 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8868092-5901-4460-8779-9f4df05079b4 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:58.232841 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 39/83] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:25:58 2022] PUT /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643/users/0b1931fa42bd48ffae39f00dd53304fb/roles/6d1894444bfd400a89591929bd8925de => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:25:58.234116 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c60073-c258-45f1-aebc-39427861ab54 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:58.311568 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:58.405773 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:58.422669 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8868092-5901-4460-8779-9f4df05079b4 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:58.437248 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c60073-c258-45f1-aebc-39427861ab54 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:58.484136 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a97fe-7c9a-434b-9e02-ac4fccbf67c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:58.484823 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a97fe-7c9a-434b-9e02-ac4fccbf67c0 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:58.485110 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a97fe-7c9a-434b-9e02-ac4fccbf67c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:58.485388 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a97fe-7c9a-434b-9e02-ac4fccbf67c0 None admin] PATH_INFO: `/v3/projects/9b80a8562be24c25afde8eb066625a32` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:58.486537 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a97fe-7c9a-434b-9e02-ac4fccbf67c0 None admin] RBAC: Authorizing `identity:get_project(project_id=9b80a8562be24c25afde8eb066625a32)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:58.488009 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a97fe-7c9a-434b-9e02-ac4fccbf67c0 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:58.489939 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 45/84] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:25:58 2022] GET /identity/v3/projects/9b80a8562be24c25afde8eb066625a32 => generated 312 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:25:58.496119 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a97fe-7c9a-434b-9e02-ac4fccbf67c0 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:25:58.497981 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc538e0-285a-449f-859d-91a4a87af534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:58.498670 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc538e0-285a-449f-859d-91a4a87af534 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:58.498946 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc538e0-285a-449f-859d-91a4a87af534 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:58.499351 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc538e0-285a-449f-859d-91a4a87af534 None admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643/users/3ce73a2078a24b19af860a9906b4d83e/roles/4b1f48ab911e4fadbc70164ae43d508b` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:58.502358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc538e0-285a-449f-859d-91a4a87af534 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad7b7e0473f64a4685d43fc69df8b643, user_id=3ce73a2078a24b19af860a9906b4d83e, role_id=4b1f48ab911e4fadbc70164ae43d508b)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:58.503910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc538e0-285a-449f-859d-91a4a87af534 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:58.533251 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 40/85] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:25:58 2022] PUT /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643/users/3ce73a2078a24b19af860a9906b4d83e/roles/4b1f48ab911e4fadbc70164ae43d508b => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:25:58.642549 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a97fe-7c9a-434b-9e02-ac4fccbf67c0 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:25:58.664117 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:25:58.687433 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15fa7a-51d3-4bfa-b1b0-d5a23d404203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:25:58.688174 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15fa7a-51d3-4bfa-b1b0-d5a23d404203 None admin] REQUEST_METHOD: `PUT` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:25:58.688456 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15fa7a-51d3-4bfa-b1b0-d5a23d404203 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:25:58.688725 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15fa7a-51d3-4bfa-b1b0-d5a23d404203 None admin] PATH_INFO: `/v3/projects/9b80a8562be24c25afde8eb066625a32/users/0b1931fa42bd48ffae39f00dd53304fb/roles/6d1894444bfd400a89591929bd8925de` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:25:58.690823 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15fa7a-51d3-4bfa-b1b0-d5a23d404203 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b80a8562be24c25afde8eb066625a32, user_id=0b1931fa42bd48ffae39f00dd53304fb, role_id=6d1894444bfd400a89591929bd8925de)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:25:58.692412 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15fa7a-51d3-4bfa-b1b0-d5a23d404203 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:25:58.708398 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 46/86] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:25:58 2022] PUT /identity/v3/projects/9b80a8562be24c25afde8eb066625a32/users/0b1931fa42bd48ffae39f00dd53304fb/roles/6d1894444bfd400a89591929bd8925de => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 14:26:00.194717 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3d28b-b774-4643-a6b1-eaa5ce146a33 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:00.195291 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3d28b-b774-4643-a6b1-eaa5ce146a33 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:00.195291 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3d28b-b774-4643-a6b1-eaa5ce146a33 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:00.197026 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 41/87] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:00.203330 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01950f01-26ac-45d4-bce3-b31923319cf5 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:00.203829 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01950f01-26ac-45d4-bce3-b31923319cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:00.203899 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01950f01-26ac-45d4-bce3-b31923319cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:00.232485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-01950f01-26ac-45d4-bce3-b31923319cf5 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:00.233678 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-01950f01-26ac-45d4-bce3-b31923319cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:00.259346 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36023316-1520-435b-a87a-d4a497595b16 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:00.259634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36023316-1520-435b-a87a-d4a497595b16 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:00.259893 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36023316-1520-435b-a87a-d4a497595b16 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:00.261223 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 42/88] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:26:00.266699 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81755b-3454-4ac3-a445-9da0354af5c7 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:00.266985 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81755b-3454-4ac3-a445-9da0354af5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:00.267314 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81755b-3454-4ac3-a445-9da0354af5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:00.288630 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-4d81755b-3454-4ac3-a445-9da0354af5c7 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:00.289829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-4d81755b-3454-4ac3-a445-9da0354af5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:00.356756 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 47/89] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:00.363632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 43/90] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:26:00.728365 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01950f01-26ac-45d4-bce3-b31923319cf5 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:00.843828 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01950f01-26ac-45d4-bce3-b31923319cf5 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:00.860125 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:00.890640 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16083849-9dcd-4031-902f-eb72e3cbcd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:00.891312 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16083849-9dcd-4031-902f-eb72e3cbcd19 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:00.891593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16083849-9dcd-4031-902f-eb72e3cbcd19 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:00.891865 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16083849-9dcd-4031-902f-eb72e3cbcd19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:00.892982 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16083849-9dcd-4031-902f-eb72e3cbcd19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:00.894430 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16083849-9dcd-4031-902f-eb72e3cbcd19 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:00.896031 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 48/91] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:00.901689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81755b-3454-4ac3-a445-9da0354af5c7 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:00.923466 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16083849-9dcd-4031-902f-eb72e3cbcd19 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:01.049017 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:01.100398 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81755b-3454-4ac3-a445-9da0354af5c7 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:01.131781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16083849-9dcd-4031-902f-eb72e3cbcd19 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:01.192848 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168d1d5-c85a-4ec9-9f6c-8e6e6896f1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:01.193468 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168d1d5-c85a-4ec9-9f6c-8e6e6896f1b4 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:01.195063 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168d1d5-c85a-4ec9-9f6c-8e6e6896f1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:01.195672 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168d1d5-c85a-4ec9-9f6c-8e6e6896f1b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:01.200883 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168d1d5-c85a-4ec9-9f6c-8e6e6896f1b4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:01.207853 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168d1d5-c85a-4ec9-9f6c-8e6e6896f1b4 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:01.228819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc3756-80fe-4d14-a836-048fce30ee6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:01.229471 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc3756-80fe-4d14-a836-048fce30ee6e None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:01.229759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc3756-80fe-4d14-a836-048fce30ee6e None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:01.230033 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc3756-80fe-4d14-a836-048fce30ee6e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:01.259901 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc3756-80fe-4d14-a836-048fce30ee6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:01.260855 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 44/92] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Oct 12 14:26:00 2022] POST /identity/v3/projects => generated 302 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:26:01.265819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-4ffc3756-80fe-4d14-a836-048fce30ee6e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 14:26:01.265819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:01.265819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:01.265819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:01.265819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:01.265819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:01.265819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:01.265819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:01.265819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:01.265819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:01.265819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:01.265819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:01.265819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:01.265819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:01.265819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:01.265819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:01.266693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:01.266693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:01.266693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:01.266693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:01.266693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:01.266693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:01.266693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:01.266693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:01.266693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:01.266693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:01.266693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:01.266693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:01.266693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:01.266693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:01.266693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:01.266693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:01.267495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:01.267495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:01.267495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:01.267495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:01.267495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:01.267495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:01.267495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:01.267495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:01.267495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:01.267495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:01.267495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 14:26:01.267495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:01.271243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 49/93] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:01.273026 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168d1d5-c85a-4ec9-9f6c-8e6e6896f1b4 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:01.399571 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168d1d5-c85a-4ec9-9f6c-8e6e6896f1b4 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:01.432480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:01.445179 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d86ac-cbc5-4d82-bb7b-f5f9c7fd45c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:01.445800 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d86ac-cbc5-4d82-bb7b-f5f9c7fd45c8 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:01.446074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d86ac-cbc5-4d82-bb7b-f5f9c7fd45c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:01.446503 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d86ac-cbc5-4d82-bb7b-f5f9c7fd45c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:01.447194 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d86ac-cbc5-4d82-bb7b-f5f9c7fd45c8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:01.447499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d86ac-cbc5-4d82-bb7b-f5f9c7fd45c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:01.448903 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d86ac-cbc5-4d82-bb7b-f5f9c7fd45c8 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:01.464433 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 45/94] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:26:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:01.476310 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc3756-80fe-4d14-a836-048fce30ee6e None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:01.533909 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:01.626327 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc3756-80fe-4d14-a836-048fce30ee6e None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:01.679203 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea9a87-3708-46f4-b270-c675f4c34274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:01.679778 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea9a87-3708-46f4-b270-c675f4c34274 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:01.680056 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea9a87-3708-46f4-b270-c675f4c34274 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:01.680326 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea9a87-3708-46f4-b270-c675f4c34274 None admin] PATH_INFO: `/v3/users/ff7e49a126924acc8a33086c095bca4d` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:01.697074 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea9a87-3708-46f4-b270-c675f4c34274 None admin] RBAC: Authorizing `identity:get_user(user_id=ff7e49a126924acc8a33086c095bca4d)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:01.698555 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea9a87-3708-46f4-b270-c675f4c34274 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:01.700350 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 50/95] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:01 2022] GET /identity/v3/users/ff7e49a126924acc8a33086c095bca4d => generated 289 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:01.708542 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d86ac-cbc5-4d82-bb7b-f5f9c7fd45c8 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:01.767096 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80901a-2566-49b5-9a04-53a94e23be2b None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:01.767757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80901a-2566-49b5-9a04-53a94e23be2b None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:01.768229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80901a-2566-49b5-9a04-53a94e23be2b None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:01.769772 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 51/96] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:26:01.776703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c0ba2-e0d0-41d7-8e74-ea9f2bd42b42 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:01.779488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c0ba2-e0d0-41d7-8e74-ea9f2bd42b42 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:01.780009 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c0ba2-e0d0-41d7-8e74-ea9f2bd42b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:01.813242 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-1e6c0ba2-e0d0-41d7-8e74-ea9f2bd42b42 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:01.815610 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-1e6c0ba2-e0d0-41d7-8e74-ea9f2bd42b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:01.884024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d86ac-cbc5-4d82-bb7b-f5f9c7fd45c8 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:01.920296 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 52/97] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:01.921951 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49735c0d-6105-4eda-b0cb-42939d089bb9 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:01.922379 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49735c0d-6105-4eda-b0cb-42939d089bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:01.922771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49735c0d-6105-4eda-b0cb-42939d089bb9 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:01.924193 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 53/98] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:01.933436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cc76f-7be9-4a45-8424-67c13c2b9299 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:01.934054 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cc76f-7be9-4a45-8424-67c13c2b9299 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:01.934645 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cc76f-7be9-4a45-8424-67c13c2b9299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:01.938375 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc496744-a866-4c80-8870-5bc12180b092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:01.939688 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc496744-a866-4c80-8870-5bc12180b092 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:01.941118 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc496744-a866-4c80-8870-5bc12180b092 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:01.941545 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc496744-a866-4c80-8870-5bc12180b092 None admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:01.943809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc496744-a866-4c80-8870-5bc12180b092 None admin] RBAC: Authorizing `identity:get_project(project_id=ad7b7e0473f64a4685d43fc69df8b643)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:01.946006 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc496744-a866-4c80-8870-5bc12180b092 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:01.948233 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 46/99] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:01 2022] GET /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:01.960449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc496744-a866-4c80-8870-5bc12180b092 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:01.977636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-743cc76f-7be9-4a45-8424-67c13c2b9299 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:01.979122 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-743cc76f-7be9-4a45-8424-67c13c2b9299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:02.008376 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:02.096124 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 54/100] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:26:02.097598 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f4086-5c90-4ae3-810f-503a04474673 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:02.098018 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f4086-5c90-4ae3-810f-503a04474673 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:02.098455 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f4086-5c90-4ae3-810f-503a04474673 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:02.099890 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 55/101] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:02.105553 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcd9c8-733e-42c0-b913-d68b3e2de548 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:02.107392 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcd9c8-733e-42c0-b913-d68b3e2de548 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:02.107802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcd9c8-733e-42c0-b913-d68b3e2de548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:02.112193 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:02.147343 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-95bcd9c8-733e-42c0-b913-d68b3e2de548 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:02.149750 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-95bcd9c8-733e-42c0-b913-d68b3e2de548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:02.201187 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc496744-a866-4c80-8870-5bc12180b092 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:02.266467 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789bc4e-e0ee-41ed-8e85-70becff98118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:02.267419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789bc4e-e0ee-41ed-8e85-70becff98118 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:02.268585 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789bc4e-e0ee-41ed-8e85-70becff98118 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:02.269002 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789bc4e-e0ee-41ed-8e85-70becff98118 None admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643/users/ff7e49a126924acc8a33086c095bca4d/roles/ac952ec5ebb24b2fa6476a7990e40147` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:02.273375 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789bc4e-e0ee-41ed-8e85-70becff98118 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad7b7e0473f64a4685d43fc69df8b643, user_id=ff7e49a126924acc8a33086c095bca4d, role_id=ac952ec5ebb24b2fa6476a7990e40147)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:02.275074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789bc4e-e0ee-41ed-8e85-70becff98118 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:02.288779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 56/102] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:26:02.291107 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7ab59-7349-4fa8-acec-e40faf746efc None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:02.291630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7ab59-7349-4fa8-acec-e40faf746efc None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:02.292040 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7ab59-7349-4fa8-acec-e40faf746efc None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:02.293670 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 57/103] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:02.299436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643b78c-fa01-406d-81e2-f9ed9a0c57b0 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:02.299729 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643b78c-fa01-406d-81e2-f9ed9a0c57b0 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:02.300049 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643b78c-fa01-406d-81e2-f9ed9a0c57b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:02.315276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 47/104] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:26:01 2022] PUT /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643/users/ff7e49a126924acc8a33086c095bca4d/roles/ac952ec5ebb24b2fa6476a7990e40147 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:26:02.329921 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-5643b78c-fa01-406d-81e2-f9ed9a0c57b0 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:02.331132 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-5643b78c-fa01-406d-81e2-f9ed9a0c57b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:02.338765 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789bc4e-e0ee-41ed-8e85-70becff98118 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:02.472487 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 58/105] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:02.473338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643b78c-fa01-406d-81e2-f9ed9a0c57b0 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:02.503000 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:02.554859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789bc4e-e0ee-41ed-8e85-70becff98118 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:02.588582 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:02.608716 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc75d8-3fd2-44cd-be8c-7cef4c39fdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:02.609362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc75d8-3fd2-44cd-be8c-7cef4c39fdf6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:02.609636 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc75d8-3fd2-44cd-be8c-7cef4c39fdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:02.609904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc75d8-3fd2-44cd-be8c-7cef4c39fdf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:02.618992 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643b78c-fa01-406d-81e2-f9ed9a0c57b0 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:02.633897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc75d8-3fd2-44cd-be8c-7cef4c39fdf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:02.640300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-35fc75d8-3fd2-44cd-be8c-7cef4c39fdf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:02.640300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:02.640300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:02.640300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:02.640300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:02.640300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:02.640300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:02.640300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:02.640300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:02.640300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:02.640300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:02.640300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:02.640300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:02.640300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:02.640300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:02.640300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:02.641135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:02.641135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:02.641135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:02.641135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:02.641135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:02.641135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:02.641135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:02.641135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:02.641135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:02.641135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:02.641135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:02.641135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:02.641135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:02.641135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:02.641135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:02.641135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:02.641862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:02.641862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:02.641862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:02.641862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:02.641862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:02.641862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:02.641862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:02.641862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:02.641862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:02.641862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:02.641862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:02.641862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:02.641862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 48/106] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:02.651701 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc75d8-3fd2-44cd-be8c-7cef4c39fdf6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:02.692516 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172a18a-0f3e-45e5-895d-b3ea8bb5df5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:02.693149 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172a18a-0f3e-45e5-895d-b3ea8bb5df5d None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:02.693422 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172a18a-0f3e-45e5-895d-b3ea8bb5df5d None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:02.693687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172a18a-0f3e-45e5-895d-b3ea8bb5df5d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:02.711597 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172a18a-0f3e-45e5-895d-b3ea8bb5df5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:02.717612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-2172a18a-0f3e-45e5-895d-b3ea8bb5df5d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:26:02.717612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:02.717612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:02.717612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:02.717612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:02.717612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:02.717612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:02.717612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:02.717612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:02.717612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:02.717612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:02.717612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:02.717612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:02.717612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:02.717612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:02.717612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:02.719703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:02.719703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:02.719703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:02.719703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:02.719703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:02.719703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:02.719703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:02.719703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:02.719703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:02.719703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:02.719703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:02.719703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:02.719703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:02.719703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:02.719703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:02.719703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:02.721579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:02.721579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:02.721579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:02.721579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:02.721579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:02.721579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:02.721579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:02.721579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:02.721579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:02.721579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:02.721579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:26:02.721579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:02.721579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 59/107] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Oct 12 14:26:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:02.728319 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172a18a-0f3e-45e5-895d-b3ea8bb5df5d None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:02.836066 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:02.859630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172a18a-0f3e-45e5-895d-b3ea8bb5df5d None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:02.871774 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc75d8-3fd2-44cd-be8c-7cef4c39fdf6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:02.908933 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4dc4b-4c00-4835-91bb-565abfa34265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:02.909541 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4dc4b-4c00-4835-91bb-565abfa34265 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:02.909817 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4dc4b-4c00-4835-91bb-565abfa34265 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:02.910086 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4dc4b-4c00-4835-91bb-565abfa34265 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:02.910751 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4dc4b-4c00-4835-91bb-565abfa34265 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:02.911050 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4dc4b-4c00-4835-91bb-565abfa34265 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:02.912486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4dc4b-4c00-4835-91bb-565abfa34265 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:02.917198 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18be91-1447-4077-8b3f-22859b7d5680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:02.918704 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18be91-1447-4077-8b3f-22859b7d5680 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:02.919274 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18be91-1447-4077-8b3f-22859b7d5680 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:02.919690 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18be91-1447-4077-8b3f-22859b7d5680 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:02.920517 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18be91-1447-4077-8b3f-22859b7d5680 None admin] RBAC: Adding query filter params (name=member) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:02.920959 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18be91-1447-4077-8b3f-22859b7d5680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:02.921948 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 60/108] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:02.922642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18be91-1447-4077-8b3f-22859b7d5680 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:02.927490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4dc4b-4c00-4835-91bb-565abfa34265 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:02.935921 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 49/109] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:26:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:02.937424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18be91-1447-4077-8b3f-22859b7d5680 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:02.988592 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:03.047205 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4dc4b-4c00-4835-91bb-565abfa34265 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:03.059006 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18be91-1447-4077-8b3f-22859b7d5680 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:03.093713 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae472bdd-8c1c-459e-a00a-e47c5a30ccd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:03.094500 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae472bdd-8c1c-459e-a00a-e47c5a30ccd2 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:03.095301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae472bdd-8c1c-459e-a00a-e47c5a30ccd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:03.095678 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae472bdd-8c1c-459e-a00a-e47c5a30ccd2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:03.098711 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:03.100671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9398ed3-2d31-4c2f-8511-387ec2cff745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:03.101547 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9398ed3-2d31-4c2f-8511-387ec2cff745 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:03.101964 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9398ed3-2d31-4c2f-8511-387ec2cff745 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:03.102743 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9398ed3-2d31-4c2f-8511-387ec2cff745 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:03.110505 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae472bdd-8c1c-459e-a00a-e47c5a30ccd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:03.114249 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9398ed3-2d31-4c2f-8511-387ec2cff745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:03.116902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-ae472bdd-8c1c-459e-a00a-e47c5a30ccd2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:03.116902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:03.116902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:03.116902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:03.116902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:03.116902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:03.116902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:03.116902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:03.116902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:03.116902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:03.116902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:03.116902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:03.116902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:03.116902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:03.116902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:03.116902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:03.117722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:03.117722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:03.117722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:03.117722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:03.117722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:03.117722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:03.117722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:03.117722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:03.117722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:03.117722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:03.117722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:03.117722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:03.117722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:03.117722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:03.117722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:03.117722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:03.118475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:03.118475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:03.118475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:03.118475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:03.118475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:03.118475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:03.118475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:03.118475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:03.118475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:03.118475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:03.118475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:03.118475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:03.118475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 61/110] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:03.119083 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae472bdd-8c1c-459e-a00a-e47c5a30ccd2 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:03.124344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-f9398ed3-2d31-4c2f-8511-387ec2cff745 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:26:03.124344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:03.124344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:03.124344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:03.124344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:03.124344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:03.124344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:03.124344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:03.124344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:03.124344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:03.124344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:03.124344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:03.124344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:03.124344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:03.124344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:03.124344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:03.125264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:03.125264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:03.125264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:03.125264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:03.125264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:03.125264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:03.125264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:03.125264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:03.125264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:03.125264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:03.125264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:03.125264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:03.125264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:03.125264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:03.125264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:03.125264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:03.125992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:03.125992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:03.125992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:03.125992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:03.125992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:03.125992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:03.125992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:03.125992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:03.125992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:03.125992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:03.125992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:26:03.125992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:03.128306 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 50/111] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:26:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:03.134348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9398ed3-2d31-4c2f-8511-387ec2cff745 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:03.235008 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae472bdd-8c1c-459e-a00a-e47c5a30ccd2 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:03.250348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9398ed3-2d31-4c2f-8511-387ec2cff745 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:03.258869 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:03.271066 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1862d-69a8-411e-be21-aadf05f8e9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:03.271822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1862d-69a8-411e-be21-aadf05f8e9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:03.272172 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1862d-69a8-411e-be21-aadf05f8e9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:03.272538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1862d-69a8-411e-be21-aadf05f8e9b2 None admin] PATH_INFO: `/v3/users/0b1931fa42bd48ffae39f00dd53304fb` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:03.273804 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1862d-69a8-411e-be21-aadf05f8e9b2 None admin] RBAC: Authorizing `identity:get_user(user_id=0b1931fa42bd48ffae39f00dd53304fb)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:03.275687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1862d-69a8-411e-be21-aadf05f8e9b2 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:03.278059 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 62/112] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:03 2022] GET /identity/v3/users/0b1931fa42bd48ffae39f00dd53304fb => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:03.283247 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1862d-69a8-411e-be21-aadf05f8e9b2 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:03.291825 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fba5c-d43d-45b5-91ac-c432dd234137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:03.292828 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fba5c-d43d-45b5-91ac-c432dd234137 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:03.293490 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fba5c-d43d-45b5-91ac-c432dd234137 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:03.293962 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fba5c-d43d-45b5-91ac-c432dd234137 None admin] PATH_INFO: `/v3/users/0b1931fa42bd48ffae39f00dd53304fb` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:03.295375 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fba5c-d43d-45b5-91ac-c432dd234137 None admin] RBAC: Authorizing `identity:get_user(user_id=0b1931fa42bd48ffae39f00dd53304fb)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:03.297098 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fba5c-d43d-45b5-91ac-c432dd234137 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:03.298886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 51/113] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:03 2022] GET /identity/v3/users/0b1931fa42bd48ffae39f00dd53304fb => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:03.299869 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fba5c-d43d-45b5-91ac-c432dd234137 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:03.348536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:03.389514 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1862d-69a8-411e-be21-aadf05f8e9b2 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:03.416574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fba5c-d43d-45b5-91ac-c432dd234137 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:03.425292 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1d5e3-e8f2-400b-9f76-50b4872789f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:03.426084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1d5e3-e8f2-400b-9f76-50b4872789f2 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:03.426457 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1d5e3-e8f2-400b-9f76-50b4872789f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:03.426865 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1d5e3-e8f2-400b-9f76-50b4872789f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:03.427720 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1d5e3-e8f2-400b-9f76-50b4872789f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:03.428061 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1d5e3-e8f2-400b-9f76-50b4872789f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:03.429558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1d5e3-e8f2-400b-9f76-50b4872789f2 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:03.438011 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 63/114] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:26:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:03.438912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1d5e3-e8f2-400b-9f76-50b4872789f2 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:03.454916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116d28c-3087-4a36-8a96-801deaa9e57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:03.455983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116d28c-3087-4a36-8a96-801deaa9e57c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:03.456409 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116d28c-3087-4a36-8a96-801deaa9e57c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:03.457261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116d28c-3087-4a36-8a96-801deaa9e57c None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:03.458350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116d28c-3087-4a36-8a96-801deaa9e57c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:03.458798 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116d28c-3087-4a36-8a96-801deaa9e57c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:03.460731 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116d28c-3087-4a36-8a96-801deaa9e57c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:03.471502 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 52/115] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:03.477278 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116d28c-3087-4a36-8a96-801deaa9e57c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:03.547630 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:03.569121 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1d5e3-e8f2-400b-9f76-50b4872789f2 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:03.596936 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116d28c-3087-4a36-8a96-801deaa9e57c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:03.603207 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:03.613892 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82454fe-fd67-48fa-9938-03abb8df50a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:03.614609 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82454fe-fd67-48fa-9938-03abb8df50a6 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:03.614932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82454fe-fd67-48fa-9938-03abb8df50a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:03.615285 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82454fe-fd67-48fa-9938-03abb8df50a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:03.644547 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b670f-6b84-4312-a1f1-fa3dd506f437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:03.646061 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b670f-6b84-4312-a1f1-fa3dd506f437 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:03.646657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b670f-6b84-4312-a1f1-fa3dd506f437 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:03.647377 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b670f-6b84-4312-a1f1-fa3dd506f437 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:03.688626 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82454fe-fd67-48fa-9938-03abb8df50a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:03.690215 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82454fe-fd67-48fa-9938-03abb8df50a6 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:03.729845 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b670f-6b84-4312-a1f1-fa3dd506f437 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:03.731926 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b670f-6b84-4312-a1f1-fa3dd506f437 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:03.769113 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:03.811770 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 64/116] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 14:26:03.813080 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82454fe-fd67-48fa-9938-03abb8df50a6 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:03.842103 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:03.853570 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 53/117] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 14:26:03.859660 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b670f-6b84-4312-a1f1-fa3dd506f437 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:03.924171 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82454fe-fd67-48fa-9938-03abb8df50a6 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:03.963110 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289afc1d-7110-4df7-8107-c913ccbf6f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:03.964108 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289afc1d-7110-4df7-8107-c913ccbf6f8a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:03.964641 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289afc1d-7110-4df7-8107-c913ccbf6f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:03.965108 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289afc1d-7110-4df7-8107-c913ccbf6f8a None admin] PATH_INFO: `/v3/users/0b1931fa42bd48ffae39f00dd53304fb` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:03.966648 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289afc1d-7110-4df7-8107-c913ccbf6f8a None admin] RBAC: Authorizing `identity:get_user(user_id=0b1931fa42bd48ffae39f00dd53304fb)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:03.968419 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289afc1d-7110-4df7-8107-c913ccbf6f8a None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:03.969922 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b670f-6b84-4312-a1f1-fa3dd506f437 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:03.970475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 65/118] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:03 2022] GET /identity/v3/users/0b1931fa42bd48ffae39f00dd53304fb => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:03.971451 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289afc1d-7110-4df7-8107-c913ccbf6f8a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:04.006775 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:04.019003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f833a28-db3a-40f4-80b3-0f1a992c744b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:04.019881 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f833a28-db3a-40f4-80b3-0f1a992c744b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:04.020207 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f833a28-db3a-40f4-80b3-0f1a992c744b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:04.020483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f833a28-db3a-40f4-80b3-0f1a992c744b None admin] PATH_INFO: `/v3/users/3ce73a2078a24b19af860a9906b4d83e` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:04.021784 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f833a28-db3a-40f4-80b3-0f1a992c744b None admin] RBAC: Authorizing `identity:get_user(user_id=3ce73a2078a24b19af860a9906b4d83e)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:04.023167 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f833a28-db3a-40f4-80b3-0f1a992c744b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:04.025025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 54/119] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:03 2022] GET /identity/v3/users/3ce73a2078a24b19af860a9906b4d83e => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:04.026254 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f833a28-db3a-40f4-80b3-0f1a992c744b None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:04.075119 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:04.083405 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289afc1d-7110-4df7-8107-c913ccbf6f8a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:04.135596 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e44c6-ad54-4d0f-9ea2-4d9304db2dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:04.136244 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e44c6-ad54-4d0f-9ea2-4d9304db2dec None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:04.137345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e44c6-ad54-4d0f-9ea2-4d9304db2dec None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:04.137621 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e44c6-ad54-4d0f-9ea2-4d9304db2dec None admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:04.139282 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e44c6-ad54-4d0f-9ea2-4d9304db2dec None admin] RBAC: Authorizing `identity:get_project(project_id=ad7b7e0473f64a4685d43fc69df8b643)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:04.141182 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e44c6-ad54-4d0f-9ea2-4d9304db2dec None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:04.143563 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 66/120] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:03 2022] GET /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:04.144358 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e44c6-ad54-4d0f-9ea2-4d9304db2dec None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:04.157028 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f833a28-db3a-40f4-80b3-0f1a992c744b None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:04.195570 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41a8ad-9de0-408d-8bf2-3b8662ada50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:04.196559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41a8ad-9de0-408d-8bf2-3b8662ada50c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:04.197057 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41a8ad-9de0-408d-8bf2-3b8662ada50c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:04.197543 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41a8ad-9de0-408d-8bf2-3b8662ada50c None admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:04.199465 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41a8ad-9de0-408d-8bf2-3b8662ada50c None admin] RBAC: Authorizing `identity:get_project(project_id=ad7b7e0473f64a4685d43fc69df8b643)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:04.201368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41a8ad-9de0-408d-8bf2-3b8662ada50c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:04.204212 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 55/121] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:04 2022] GET /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:04.205327 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41a8ad-9de0-408d-8bf2-3b8662ada50c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:04.271254 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e44c6-ad54-4d0f-9ea2-4d9304db2dec None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:04.277986 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:04.307497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95289da0-2c46-4675-8ec7-93ccc3fd0aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:04.308191 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95289da0-2c46-4675-8ec7-93ccc3fd0aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:04.308953 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95289da0-2c46-4675-8ec7-93ccc3fd0aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:04.309228 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95289da0-2c46-4675-8ec7-93ccc3fd0aa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:04.316695 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41a8ad-9de0-408d-8bf2-3b8662ada50c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:04.342526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:04.352130 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb97fa2-2396-4208-81f9-a636e2a0505e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:04.352752 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb97fa2-2396-4208-81f9-a636e2a0505e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:04.353037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb97fa2-2396-4208-81f9-a636e2a0505e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:04.353309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb97fa2-2396-4208-81f9-a636e2a0505e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:04.385155 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95289da0-2c46-4675-8ec7-93ccc3fd0aa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:04.387190 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95289da0-2c46-4675-8ec7-93ccc3fd0aa3 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:04.427917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb97fa2-2396-4208-81f9-a636e2a0505e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:04.429386 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb97fa2-2396-4208-81f9-a636e2a0505e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:04.502042 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 67/122] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 14:26:04.502042 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95289da0-2c46-4675-8ec7-93ccc3fd0aa3 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:04.505066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:04.546371 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 56/123] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 14:26:04.551693 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb97fa2-2396-4208-81f9-a636e2a0505e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:04.562081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:04.606132 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95289da0-2c46-4675-8ec7-93ccc3fd0aa3 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:04.643459 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9630ca5-7f0e-4da4-a851-abd8335977a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:04.644173 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9630ca5-7f0e-4da4-a851-abd8335977a4 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:04.644469 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9630ca5-7f0e-4da4-a851-abd8335977a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:04.644819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9630ca5-7f0e-4da4-a851-abd8335977a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:04.645549 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9630ca5-7f0e-4da4-a851-abd8335977a4 None admin] RBAC: Adding query filter params (role.id=f0761dd38b794ee6bfab00ac01f240cd, scope.project.id=ad7b7e0473f64a4685d43fc69df8b643, user.id=0b1931fa42bd48ffae39f00dd53304fb) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:04.645855 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9630ca5-7f0e-4da4-a851-abd8335977a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:04.647499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9630ca5-7f0e-4da4-a851-abd8335977a4 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:04.657259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 68/124] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:26:04 2022] GET /identity/v3/role_assignments?user.id=0b1931fa42bd48ffae39f00dd53304fb&scope.project.id=ad7b7e0473f64a4685d43fc69df8b643&role.id=f0761dd38b794ee6bfab00ac01f240cd => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:04.658385 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188d1f2-92ba-4799-8921-7262409ff7ce None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:04.658668 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188d1f2-92ba-4799-8921-7262409ff7ce None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:04.658944 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188d1f2-92ba-4799-8921-7262409ff7ce None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:04.660561 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 69/125] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:04.661204 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188d1f2-92ba-4799-8921-7262409ff7ce None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:04.676398 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb97fa2-2396-4208-81f9-a636e2a0505e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:04.728113 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305f6c9-b838-439a-92f9-14b7f3cfb805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:04.728742 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305f6c9-b838-439a-92f9-14b7f3cfb805 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:04.729016 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305f6c9-b838-439a-92f9-14b7f3cfb805 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:04.729287 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305f6c9-b838-439a-92f9-14b7f3cfb805 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:04.730109 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305f6c9-b838-439a-92f9-14b7f3cfb805 None admin] RBAC: Adding query filter params (role.id=6d1894444bfd400a89591929bd8925de, scope.project.id=ad7b7e0473f64a4685d43fc69df8b643, user.id=0b1931fa42bd48ffae39f00dd53304fb) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:04.730412 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305f6c9-b838-439a-92f9-14b7f3cfb805 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:04.732048 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305f6c9-b838-439a-92f9-14b7f3cfb805 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:04.739459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 57/126] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:26:04 2022] GET /identity/v3/role_assignments?user.id=0b1931fa42bd48ffae39f00dd53304fb&scope.project.id=ad7b7e0473f64a4685d43fc69df8b643&role.id=6d1894444bfd400a89591929bd8925de => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:04.744543 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305f6c9-b838-439a-92f9-14b7f3cfb805 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:04.797244 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188d1f2-92ba-4799-8921-7262409ff7ce None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:04.813307 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:04.841858 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:04.854591 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0063c-9ce8-4d45-82d9-6149b7184e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:04.855674 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0063c-9ce8-4d45-82d9-6149b7184e56 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:04.855961 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0063c-9ce8-4d45-82d9-6149b7184e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:04.856292 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0063c-9ce8-4d45-82d9-6149b7184e56 None admin] PATH_INFO: `/v3/projects/9b80a8562be24c25afde8eb066625a32` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:04.858042 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0063c-9ce8-4d45-82d9-6149b7184e56 None admin] RBAC: Authorizing `identity:get_project(project_id=9b80a8562be24c25afde8eb066625a32)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:04.860039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0063c-9ce8-4d45-82d9-6149b7184e56 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:04.862379 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 70/127] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:04 2022] GET /identity/v3/projects/9b80a8562be24c25afde8eb066625a32 => generated 312 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:04.864481 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c5a5f-1035-415d-b86d-6e12b926c726 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:04.864481 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c5a5f-1035-415d-b86d-6e12b926c726 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:04.864481 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c5a5f-1035-415d-b86d-6e12b926c726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:04.895353 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-170c5a5f-1035-415d-b86d-6e12b926c726 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:04.896908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-170c5a5f-1035-415d-b86d-6e12b926c726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:04.920163 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305f6c9-b838-439a-92f9-14b7f3cfb805 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:04.959730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dc232-9767-411b-9262-f108b05668fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:04.960371 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dc232-9767-411b-9262-f108b05668fa None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:04.960647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dc232-9767-411b-9262-f108b05668fa None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:04.960916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dc232-9767-411b-9262-f108b05668fa None admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:04.962036 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dc232-9767-411b-9262-f108b05668fa None admin] RBAC: Authorizing `identity:get_project(project_id=ad7b7e0473f64a4685d43fc69df8b643)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:04.963503 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dc232-9767-411b-9262-f108b05668fa None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:04.965639 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 58/128] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:04 2022] GET /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:04.966526 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff302e7-d57b-46c4-a44a-91b43098ad64 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:04.966809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff302e7-d57b-46c4-a44a-91b43098ad64 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:04.967082 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff302e7-d57b-46c4-a44a-91b43098ad64 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:04.968969 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 59/129] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:04.969658 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff302e7-d57b-46c4-a44a-91b43098ad64 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:04.987129 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 71/130] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:04.987999 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c5a5f-1035-415d-b86d-6e12b926c726 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:05.093929 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:05.098632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff302e7-d57b-46c4-a44a-91b43098ad64 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:05.100978 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c5a5f-1035-415d-b86d-6e12b926c726 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:05.127414 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:05.139258 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67becd10-ad61-4026-85c1-04acc4929d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:05.139603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d3451-bab4-4ebe-a36e-5c65966bea63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:05.139871 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67becd10-ad61-4026-85c1-04acc4929d27 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:05.140147 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67becd10-ad61-4026-85c1-04acc4929d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:05.140241 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d3451-bab4-4ebe-a36e-5c65966bea63 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:05.140415 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67becd10-ad61-4026-85c1-04acc4929d27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:05.140504 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d3451-bab4-4ebe-a36e-5c65966bea63 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:05.140732 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d3451-bab4-4ebe-a36e-5c65966bea63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:05.141131 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67becd10-ad61-4026-85c1-04acc4929d27 None admin] RBAC: Adding query filter params (role.id=6d1894444bfd400a89591929bd8925de, scope.project.id=9b80a8562be24c25afde8eb066625a32, user.id=0b1931fa42bd48ffae39f00dd53304fb) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:05.141436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67becd10-ad61-4026-85c1-04acc4929d27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:05.141523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d3451-bab4-4ebe-a36e-5c65966bea63 None admin] RBAC: Adding query filter params (role.id=4b1f48ab911e4fadbc70164ae43d508b, scope.project.id=ad7b7e0473f64a4685d43fc69df8b643, user.id=3ce73a2078a24b19af860a9906b4d83e) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:05.141750 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d3451-bab4-4ebe-a36e-5c65966bea63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:05.143199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67becd10-ad61-4026-85c1-04acc4929d27 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:05.143290 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d3451-bab4-4ebe-a36e-5c65966bea63 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:05.155506 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 72/131] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:26:04 2022] GET /identity/v3/role_assignments?user.id=3ce73a2078a24b19af860a9906b4d83e&scope.project.id=ad7b7e0473f64a4685d43fc69df8b643&role.id=4b1f48ab911e4fadbc70164ae43d508b => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:05.155697 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 60/132] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:26:04 2022] GET /identity/v3/role_assignments?user.id=0b1931fa42bd48ffae39f00dd53304fb&scope.project.id=9b80a8562be24c25afde8eb066625a32&role.id=6d1894444bfd400a89591929bd8925de => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:05.155789 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40ea7b-b137-4896-b6f7-f474c4561640 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:05.155789 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40ea7b-b137-4896-b6f7-f474c4561640 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:05.155789 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40ea7b-b137-4896-b6f7-f474c4561640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:05.181251 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-ca40ea7b-b137-4896-b6f7-f474c4561640 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:05.182737 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-ca40ea7b-b137-4896-b6f7-f474c4561640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:05.268080 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 73/133] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:05.277930 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67becd10-ad61-4026-85c1-04acc4929d27 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:05.397540 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67becd10-ad61-4026-85c1-04acc4929d27 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:05.433474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1340c09-cc38-41fb-9250-d25d23d208e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:05.434286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1340c09-cc38-41fb-9250-d25d23d208e2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:05.434612 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1340c09-cc38-41fb-9250-d25d23d208e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:05.434886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1340c09-cc38-41fb-9250-d25d23d208e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:05.436067 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1340c09-cc38-41fb-9250-d25d23d208e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:05.437512 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1340c09-cc38-41fb-9250-d25d23d208e2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:05.439290 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 61/134] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:05.444867 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40ea7b-b137-4896-b6f7-f474c4561640 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:05.546534 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40ea7b-b137-4896-b6f7-f474c4561640 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:05.561826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1340c09-cc38-41fb-9250-d25d23d208e2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:05.564411 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:05.585305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d8d12-14b6-4c7d-a85e-0fc785e5154a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:05.585918 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d8d12-14b6-4c7d-a85e-0fc785e5154a None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:05.586197 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d8d12-14b6-4c7d-a85e-0fc785e5154a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:05.586470 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d8d12-14b6-4c7d-a85e-0fc785e5154a None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:05.587122 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d8d12-14b6-4c7d-a85e-0fc785e5154a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:05.588679 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d8d12-14b6-4c7d-a85e-0fc785e5154a None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:05.596258 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:05.615590 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 74/135] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:26:05 2022] POST /identity/v3/users => generated 283 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:26:05.668843 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1340c09-cc38-41fb-9250-d25d23d208e2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:05.704827 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0a601-be7c-4e9c-9199-d2cb41cf35a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:05.705482 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0a601-be7c-4e9c-9199-d2cb41cf35a3 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:05.705763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0a601-be7c-4e9c-9199-d2cb41cf35a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:05.706038 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0a601-be7c-4e9c-9199-d2cb41cf35a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:05.717645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0a601-be7c-4e9c-9199-d2cb41cf35a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:05.723178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-55d0a601-be7c-4e9c-9199-d2cb41cf35a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 14:26:05.723178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:05.723178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:05.723178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:05.723178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:05.723178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:05.723178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:05.723178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:05.723178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:05.723178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:05.723178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:05.723178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:05.723178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:05.723178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:05.723178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:05.723178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:05.724108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:05.724108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:05.724108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:05.724108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:05.724108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:05.724108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:05.724108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:05.724108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:05.724108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:05.724108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:05.724108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:05.724108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:05.724108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:05.724108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:05.724108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:05.724108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:05.724887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:05.724887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:05.724887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:05.724887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:05.724887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:05.724887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:05.724887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:05.724887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:05.724887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:05.724887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:05.724887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 14:26:05.724887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:05.724887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 62/136] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:05.730434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d8d12-14b6-4c7d-a85e-0fc785e5154a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:05.833858 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d8d12-14b6-4c7d-a85e-0fc785e5154a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:05.867466 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d4d58-7e01-4736-b905-ddb7e6fad6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:05.868084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d4d58-7e01-4736-b905-ddb7e6fad6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:05.868363 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d4d58-7e01-4736-b905-ddb7e6fad6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:05.868632 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d4d58-7e01-4736-b905-ddb7e6fad6c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:05.869304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d4d58-7e01-4736-b905-ddb7e6fad6c3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:05.869602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d4d58-7e01-4736-b905-ddb7e6fad6c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:05.871100 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d4d58-7e01-4736-b905-ddb7e6fad6c3 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:05.880123 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 75/137] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:26:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:05.885726 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0a601-be7c-4e9c-9199-d2cb41cf35a3 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:05.984478 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:05.996873 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0a601-be7c-4e9c-9199-d2cb41cf35a3 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:06.038436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97707c3e-23f8-499a-b328-966f9a56b10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:06.039626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97707c3e-23f8-499a-b328-966f9a56b10a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:06.040146 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97707c3e-23f8-499a-b328-966f9a56b10a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:06.040637 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97707c3e-23f8-499a-b328-966f9a56b10a None admin] PATH_INFO: `/v3/users/ff7e49a126924acc8a33086c095bca4d` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:06.042063 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97707c3e-23f8-499a-b328-966f9a56b10a None admin] RBAC: Authorizing `identity:get_user(user_id=ff7e49a126924acc8a33086c095bca4d)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:06.044811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97707c3e-23f8-499a-b328-966f9a56b10a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:06.046644 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 63/138] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:05 2022] GET /identity/v3/users/ff7e49a126924acc8a33086c095bca4d => generated 289 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:06.052502 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d4d58-7e01-4736-b905-ddb7e6fad6c3 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:06.093681 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:06.162993 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d4d58-7e01-4736-b905-ddb7e6fad6c3 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:06.199309 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57ec68-8cd8-44c6-a99c-9055a0b6febf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:06.199930 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57ec68-8cd8-44c6-a99c-9055a0b6febf None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:06.200206 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57ec68-8cd8-44c6-a99c-9055a0b6febf None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:06.200477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57ec68-8cd8-44c6-a99c-9055a0b6febf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:06.270526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57ec68-8cd8-44c6-a99c-9055a0b6febf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:06.271960 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57ec68-8cd8-44c6-a99c-9055a0b6febf None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:06.307222 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:06.374094 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 76/139] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 14:26:06.380321 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97707c3e-23f8-499a-b328-966f9a56b10a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:06.493855 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97707c3e-23f8-499a-b328-966f9a56b10a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:06.532372 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292155a-e2d0-4f95-a773-6e91b7f9ffd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:06.533370 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292155a-e2d0-4f95-a773-6e91b7f9ffd2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:06.533850 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292155a-e2d0-4f95-a773-6e91b7f9ffd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:06.534320 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292155a-e2d0-4f95-a773-6e91b7f9ffd2 None admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:06.535699 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292155a-e2d0-4f95-a773-6e91b7f9ffd2 None admin] RBAC: Authorizing `identity:get_project(project_id=ad7b7e0473f64a4685d43fc69df8b643)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:06.537356 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292155a-e2d0-4f95-a773-6e91b7f9ffd2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:06.539217 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 64/140] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:06 2022] GET /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:06.547191 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57ec68-8cd8-44c6-a99c-9055a0b6febf None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:06.659789 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57ec68-8cd8-44c6-a99c-9055a0b6febf None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:06.697127 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce40df4-3556-45e7-94ec-86ef2d3c6fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:06.698129 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce40df4-3556-45e7-94ec-86ef2d3c6fea None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:06.699451 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce40df4-3556-45e7-94ec-86ef2d3c6fea None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:06.699937 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce40df4-3556-45e7-94ec-86ef2d3c6fea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:06.700874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce40df4-3556-45e7-94ec-86ef2d3c6fea None admin] RBAC: Adding query filter params (role.id=ac952ec5ebb24b2fa6476a7990e40147, scope.project.id=ad7b7e0473f64a4685d43fc69df8b643, user.id=ff7e49a126924acc8a33086c095bca4d) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:06.701387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce40df4-3556-45e7-94ec-86ef2d3c6fea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:06.703047 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce40df4-3556-45e7-94ec-86ef2d3c6fea None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:06.706530 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:06.709962 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 77/141] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:26:06 2022] GET /identity/v3/role_assignments?user.id=ff7e49a126924acc8a33086c095bca4d&scope.project.id=ad7b7e0473f64a4685d43fc69df8b643&role.id=ac952ec5ebb24b2fa6476a7990e40147 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:07.877162 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8777c60-7080-4f1a-a39f-f07a211a040a None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:07.877643 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8777c60-7080-4f1a-a39f-f07a211a040a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:07.877643 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8777c60-7080-4f1a-a39f-f07a211a040a None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:07.879024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 65/142] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:07.884363 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c9065-8850-4e75-b0b4-c9cce826452e None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:07.884776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c9065-8850-4e75-b0b4-c9cce826452e None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:07.884937 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c9065-8850-4e75-b0b4-c9cce826452e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:07.904664 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-ab2c9065-8850-4e75-b0b4-c9cce826452e None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:07.905856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-ab2c9065-8850-4e75-b0b4-c9cce826452e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:07.975524 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 78/143] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:08.075739 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e763bc2-b6c9-413a-af2b-7dfee60f4d85 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:08.076043 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e763bc2-b6c9-413a-af2b-7dfee60f4d85 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:08.076286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e763bc2-b6c9-413a-af2b-7dfee60f4d85 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:08.077758 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 66/144] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:08.082623 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd67ed-a57c-434b-9796-b1ee268b36f1 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:08.082927 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd67ed-a57c-434b-9796-b1ee268b36f1 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:08.083203 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd67ed-a57c-434b-9796-b1ee268b36f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:08.101570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-4ffd67ed-a57c-434b-9796-b1ee268b36f1 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:08.102718 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-4ffd67ed-a57c-434b-9796-b1ee268b36f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:08.152881 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275b23b-3dbb-4351-b198-9ed1fc8495aa None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:08.153178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275b23b-3dbb-4351-b198-9ed1fc8495aa None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:08.153440 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275b23b-3dbb-4351-b198-9ed1fc8495aa None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:08.154782 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 67/145] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:26:08.159564 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e929371-3d27-4dab-afc9-9118f4277874 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:08.159850 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e929371-3d27-4dab-afc9-9118f4277874 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:08.160125 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e929371-3d27-4dab-afc9-9118f4277874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:08.172289 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 79/146] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:08.178425 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-7e929371-3d27-4dab-afc9-9118f4277874 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:08.179637 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7e929371-3d27-4dab-afc9-9118f4277874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:08.204644 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:08.249477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 68/147] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:26:08.249947 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd67ed-a57c-434b-9796-b1ee268b36f1 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:08.310150 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:08.351429 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd67ed-a57c-434b-9796-b1ee268b36f1 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:08.385014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35317828-a352-4cce-b121-f5a84146bc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:08.385927 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35317828-a352-4cce-b121-f5a84146bc29 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:08.386203 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35317828-a352-4cce-b121-f5a84146bc29 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:08.386472 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35317828-a352-4cce-b121-f5a84146bc29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:08.396242 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35317828-a352-4cce-b121-f5a84146bc29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:08.401983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-35317828-a352-4cce-b121-f5a84146bc29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:26:08.401983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:08.401983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:08.401983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:08.401983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:08.401983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:08.401983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:08.401983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:08.401983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:08.401983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:08.401983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:08.401983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:08.401983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:08.401983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:08.401983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:08.401983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:08.403259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:08.403259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:08.403259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:08.403259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:08.403259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:08.403259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:08.403259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:08.403259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:08.403259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:08.403259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:08.403259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:08.403259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:08.403259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:08.403259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:08.403259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:08.403259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:08.404005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:08.404005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:08.404005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:08.404005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:08.404005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:08.404005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:08.404005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:08.404005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:08.404005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:08.404005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:08.404005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:26:08.404005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:08.404005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 80/148] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:26:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:08.409016 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e929371-3d27-4dab-afc9-9118f4277874 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:08.442838 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35317828-a352-4cce-b121-f5a84146bc29 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:08.510877 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e929371-3d27-4dab-afc9-9118f4277874 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:08.547165 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35317828-a352-4cce-b121-f5a84146bc29 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:08.553859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876e44f-b7ae-4e5c-92c7-85a9ea753b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:08.554468 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876e44f-b7ae-4e5c-92c7-85a9ea753b1f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:08.554742 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876e44f-b7ae-4e5c-92c7-85a9ea753b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:08.555011 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876e44f-b7ae-4e5c-92c7-85a9ea753b1f None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:08.555699 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2876e44f-b7ae-4e5c-92c7-85a9ea753b1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:08.555999 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2876e44f-b7ae-4e5c-92c7-85a9ea753b1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:08.557373 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2876e44f-b7ae-4e5c-92c7-85a9ea753b1f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:08.565860 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 69/149] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:08.566564 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876e44f-b7ae-4e5c-92c7-85a9ea753b1f None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:08.567029 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:08.588076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:08.589306 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e63ab2-b6b1-4c1c-89f0-602036f0467b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:08.589967 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e63ab2-b6b1-4c1c-89f0-602036f0467b None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:08.590204 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e63ab2-b6b1-4c1c-89f0-602036f0467b None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:08.590471 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e63ab2-b6b1-4c1c-89f0-602036f0467b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:08.600621 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e63ab2-b6b1-4c1c-89f0-602036f0467b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:08.606229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-f4e63ab2-b6b1-4c1c-89f0-602036f0467b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:26:08.606229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:08.606229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:08.606229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:08.606229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:08.606229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:08.606229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:08.606229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:08.606229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:08.606229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:08.606229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:08.606229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:08.606229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:08.606229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:08.606229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:08.606229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:08.607297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:08.607297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:08.607297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:08.607297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:08.607297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:08.607297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:08.607297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:08.607297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:08.607297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:08.607297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:08.607297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:08.607297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:08.607297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:08.607297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:08.607297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:08.607297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:08.608271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:08.608271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:08.608271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:08.608271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:08.608271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:08.608271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:08.608271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:08.608271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:08.608271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:08.608271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:08.608271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:26:08.608271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:08.608271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 81/150] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Oct 12 14:26:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:08.608271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e63ab2-b6b1-4c1c-89f0-602036f0467b None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:08.670288 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876e44f-b7ae-4e5c-92c7-85a9ea753b1f None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:08.703733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e455660-2177-4987-80be-87586527e056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:08.704358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e455660-2177-4987-80be-87586527e056 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:08.704459 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e63ab2-b6b1-4c1c-89f0-602036f0467b None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:08.704633 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e455660-2177-4987-80be-87586527e056 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:08.704902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e455660-2177-4987-80be-87586527e056 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:08.705992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e455660-2177-4987-80be-87586527e056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:08.707449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e455660-2177-4987-80be-87586527e056 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:08.709024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 70/151] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:08.709778 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e455660-2177-4987-80be-87586527e056 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:08.738854 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd860e02-2399-4ddc-9f1e-2a63cc0e0f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:08.739490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd860e02-2399-4ddc-9f1e-2a63cc0e0f3a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:08.739763 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd860e02-2399-4ddc-9f1e-2a63cc0e0f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:08.740031 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd860e02-2399-4ddc-9f1e-2a63cc0e0f3a None admin] PATH_INFO: `/v3/users/c013522a3a114b4ab2b59f659a080ee6` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:08.753836 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd860e02-2399-4ddc-9f1e-2a63cc0e0f3a None admin] RBAC: Authorizing `identity:get_user(user_id=c013522a3a114b4ab2b59f659a080ee6)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:08.755329 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd860e02-2399-4ddc-9f1e-2a63cc0e0f3a None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:08.761346 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 82/152] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:08 2022] GET /identity/v3/users/c013522a3a114b4ab2b59f659a080ee6 => generated 283 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:08.761346 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd860e02-2399-4ddc-9f1e-2a63cc0e0f3a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:08.797138 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:08.798149 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:08.811240 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e455660-2177-4987-80be-87586527e056 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:08.843081 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551113b7-92fd-49bb-9858-9c1569a70156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:08.843722 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551113b7-92fd-49bb-9858-9c1569a70156 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:08.843997 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551113b7-92fd-49bb-9858-9c1569a70156 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:08.844265 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551113b7-92fd-49bb-9858-9c1569a70156 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:08.844913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551113b7-92fd-49bb-9858-9c1569a70156 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:08.845210 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551113b7-92fd-49bb-9858-9c1569a70156 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:08.846571 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551113b7-92fd-49bb-9858-9c1569a70156 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:08.854146 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 71/153] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:08.855495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551113b7-92fd-49bb-9858-9c1569a70156 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:08.855727 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd860e02-2399-4ddc-9f1e-2a63cc0e0f3a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:08.889252 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2e57b-441a-4e5b-aa17-722cfcd8b3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:08.889878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2e57b-441a-4e5b-aa17-722cfcd8b3e7 None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:08.890110 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2e57b-441a-4e5b-aa17-722cfcd8b3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:08.890371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2e57b-441a-4e5b-aa17-722cfcd8b3e7 None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:08.891009 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2e57b-441a-4e5b-aa17-722cfcd8b3e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:08.892471 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2e57b-441a-4e5b-aa17-722cfcd8b3e7 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:08.915253 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 83/154] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:26:08 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:26:08.920023 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2e57b-441a-4e5b-aa17-722cfcd8b3e7 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:08.972515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551113b7-92fd-49bb-9858-9c1569a70156 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:09.013607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86a5ca-a4af-4140-8924-f060fc16e2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:09.014264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86a5ca-a4af-4140-8924-f060fc16e2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:09.014546 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86a5ca-a4af-4140-8924-f060fc16e2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:09.014819 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86a5ca-a4af-4140-8924-f060fc16e2b1 None admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:09.016019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86a5ca-a4af-4140-8924-f060fc16e2b1 None admin] RBAC: Authorizing `identity:get_project(project_id=a78b76de457545f7927d98a7a89f4cec)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:09.017478 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86a5ca-a4af-4140-8924-f060fc16e2b1 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:09.019206 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 72/155] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:08 2022] GET /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:09.021714 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:09.025470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86a5ca-a4af-4140-8924-f060fc16e2b1 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:09.032829 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2e57b-441a-4e5b-aa17-722cfcd8b3e7 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:09.047076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:09.075448 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338ab32-6f5c-4df9-a1a5-b5a41cde7c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:09.076257 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338ab32-6f5c-4df9-a1a5-b5a41cde7c66 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:09.076632 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338ab32-6f5c-4df9-a1a5-b5a41cde7c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:09.077001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338ab32-6f5c-4df9-a1a5-b5a41cde7c66 None admin] PATH_INFO: `/v3/users/c013522a3a114b4ab2b59f659a080ee6` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:09.078301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338ab32-6f5c-4df9-a1a5-b5a41cde7c66 None admin] RBAC: Authorizing `identity:get_user(user_id=c013522a3a114b4ab2b59f659a080ee6)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:09.079945 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338ab32-6f5c-4df9-a1a5-b5a41cde7c66 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:09.082178 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 84/156] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:08 2022] GET /identity/v3/users/c013522a3a114b4ab2b59f659a080ee6 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:09.087494 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338ab32-6f5c-4df9-a1a5-b5a41cde7c66 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:09.144117 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86a5ca-a4af-4140-8924-f060fc16e2b1 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:09.186305 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79344ff0-21f4-449a-a29e-0e6a029b804f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:09.186978 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79344ff0-21f4-449a-a29e-0e6a029b804f None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:09.187310 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79344ff0-21f4-449a-a29e-0e6a029b804f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:09.187590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79344ff0-21f4-449a-a29e-0e6a029b804f None admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec/users/c013522a3a114b4ab2b59f659a080ee6/roles/4b1f48ab911e4fadbc70164ae43d508b` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:09.189794 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79344ff0-21f4-449a-a29e-0e6a029b804f None admin] RBAC: Authorizing `identity:create_grant(project_id=a78b76de457545f7927d98a7a89f4cec, user_id=c013522a3a114b4ab2b59f659a080ee6, role_id=4b1f48ab911e4fadbc70164ae43d508b)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:09.191561 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79344ff0-21f4-449a-a29e-0e6a029b804f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:09.208410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 73/157] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:26:09 2022] PUT /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec/users/c013522a3a114b4ab2b59f659a080ee6/roles/4b1f48ab911e4fadbc70164ae43d508b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 14:26:09.211267 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338ab32-6f5c-4df9-a1a5-b5a41cde7c66 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:09.255131 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28d813-24a3-478b-b3ae-fe96778fc9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:09.255732 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28d813-24a3-478b-b3ae-fe96778fc9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:09.256006 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28d813-24a3-478b-b3ae-fe96778fc9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:09.256277 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28d813-24a3-478b-b3ae-fe96778fc9f2 None admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:09.257456 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28d813-24a3-478b-b3ae-fe96778fc9f2 None admin] RBAC: Authorizing `identity:get_project(project_id=a78b76de457545f7927d98a7a89f4cec)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:09.258925 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28d813-24a3-478b-b3ae-fe96778fc9f2 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:09.261634 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 85/158] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:09 2022] GET /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:09.267629 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79344ff0-21f4-449a-a29e-0e6a029b804f None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:09.362712 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:09.389292 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79344ff0-21f4-449a-a29e-0e6a029b804f None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:09.427360 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08ae1f-983c-4dde-bc38-b99636079d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:09.428030 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08ae1f-983c-4dde-bc38-b99636079d34 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:09.428310 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08ae1f-983c-4dde-bc38-b99636079d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:09.428592 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08ae1f-983c-4dde-bc38-b99636079d34 None admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec/users/c013522a3a114b4ab2b59f659a080ee6/roles/f0761dd38b794ee6bfab00ac01f240cd` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:09.430762 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08ae1f-983c-4dde-bc38-b99636079d34 None admin] RBAC: Authorizing `identity:create_grant(project_id=a78b76de457545f7927d98a7a89f4cec, user_id=c013522a3a114b4ab2b59f659a080ee6, role_id=f0761dd38b794ee6bfab00ac01f240cd)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:09.432250 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08ae1f-983c-4dde-bc38-b99636079d34 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:09.449761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 74/159] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:26:09 2022] PUT /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec/users/c013522a3a114b4ab2b59f659a080ee6/roles/f0761dd38b794ee6bfab00ac01f240cd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:26:11.261102 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f5843-bbb4-4bbc-b64c-b052166253f7 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:11.261650 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f5843-bbb4-4bbc-b64c-b052166253f7 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:11.261650 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f5843-bbb4-4bbc-b64c-b052166253f7 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:11.263005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 86/160] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:11.268221 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1443e9-f8ff-4061-b779-c6f9d6b6934f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:11.268617 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1443e9-f8ff-4061-b779-c6f9d6b6934f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:11.268800 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1443e9-f8ff-4061-b779-c6f9d6b6934f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:11.287795 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-5e1443e9-f8ff-4061-b779-c6f9d6b6934f None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:11.288961 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-5e1443e9-f8ff-4061-b779-c6f9d6b6934f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:11.341188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c75fd-eff8-40d3-97ea-94e3d3e6c422 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:11.341496 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c75fd-eff8-40d3-97ea-94e3d3e6c422 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:11.341736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c75fd-eff8-40d3-97ea-94e3d3e6c422 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:11.343119 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 87/161] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:26:11.348741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9988cde-59c5-4bf7-b737-9154f5311934 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:11.349025 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9988cde-59c5-4bf7-b737-9154f5311934 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:11.349296 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9988cde-59c5-4bf7-b737-9154f5311934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:11.368296 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-e9988cde-59c5-4bf7-b737-9154f5311934 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:11.370241 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-e9988cde-59c5-4bf7-b737-9154f5311934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:11.370777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 75/162] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:11.395484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:11.442445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 88/163] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:26:11.444153 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c612d08-cb70-406d-ab45-8e93460a069f None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:11.444439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c612d08-cb70-406d-ab45-8e93460a069f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:11.444712 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c612d08-cb70-406d-ab45-8e93460a069f None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:11.446163 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 76/164] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:11.451789 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af855f6-4e0f-4034-9143-2bfb6fda0d47 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:11.452072 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af855f6-4e0f-4034-9143-2bfb6fda0d47 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:11.452343 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af855f6-4e0f-4034-9143-2bfb6fda0d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:11.470871 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-2af855f6-4e0f-4034-9143-2bfb6fda0d47 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:11.472174 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-2af855f6-4e0f-4034-9143-2bfb6fda0d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:11.543008 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 89/165] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:26:11.650674 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fd058-68ef-4dfe-a5b0-85f194c873ca None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:11.651082 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fd058-68ef-4dfe-a5b0-85f194c873ca None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:11.651533 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fd058-68ef-4dfe-a5b0-85f194c873ca None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:11.653058 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 77/166] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:11.658316 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2091e06-82a0-4722-b252-caf201f95764 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:11.658793 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2091e06-82a0-4722-b252-caf201f95764 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:11.659274 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2091e06-82a0-4722-b252-caf201f95764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:11.671113 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21fd058-68ef-4dfe-a5b0-85f194c873ca None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:11.679035 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-c2091e06-82a0-4722-b252-caf201f95764 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:11.680435 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-c2091e06-82a0-4722-b252-caf201f95764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:11.742128 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:11.756766 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 90/167] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:11.758084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2091e06-82a0-4722-b252-caf201f95764 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:11.782237 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:11.791018 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21fd058-68ef-4dfe-a5b0-85f194c873ca None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:11.835486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc314f-15c2-4be6-8327-7e26e0ef9a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:11.836041 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc314f-15c2-4be6-8327-7e26e0ef9a53 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:11.836316 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc314f-15c2-4be6-8327-7e26e0ef9a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:11.836582 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc314f-15c2-4be6-8327-7e26e0ef9a53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:11.847135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc314f-15c2-4be6-8327-7e26e0ef9a53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:11.851993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-96cc314f-15c2-4be6-8327-7e26e0ef9a53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:11.851993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:11.851993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:11.851993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:11.851993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:11.851993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:11.851993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:11.851993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:11.851993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:11.851993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:11.851993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:11.851993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:11.851993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:11.851993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:11.851993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:11.851993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:11.852910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:11.852910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:11.852910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:11.852910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:11.852910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:11.852910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:11.852910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:11.852910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:11.852910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:11.852910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:11.852910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:11.852910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:11.852910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:11.852910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:11.852910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:11.852910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:11.853669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:11.853669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:11.853669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:11.853669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:11.853669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:11.853669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:11.853669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:11.853669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:11.853669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:11.853669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:11.853669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:11.853669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:11.853669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 78/168] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:11.859273 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc314f-15c2-4be6-8327-7e26e0ef9a53 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:11.867228 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2091e06-82a0-4722-b252-caf201f95764 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:11.901727 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4afe1e-5dd3-4ac5-8d78-748c898d057f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:11.902449 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4afe1e-5dd3-4ac5-8d78-748c898d057f None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:11.902728 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4afe1e-5dd3-4ac5-8d78-748c898d057f None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:11.902991 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4afe1e-5dd3-4ac5-8d78-748c898d057f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:11.904138 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4afe1e-5dd3-4ac5-8d78-748c898d057f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:11.905585 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4afe1e-5dd3-4ac5-8d78-748c898d057f None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:11.911437 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 91/169] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:11.911437 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4afe1e-5dd3-4ac5-8d78-748c898d057f None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:11.975770 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc314f-15c2-4be6-8327-7e26e0ef9a53 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:11.998717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:12.003617 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:12.012286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746e1fe-5dc5-423d-ba97-3d5ff4334ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:12.012514 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4afe1e-5dd3-4ac5-8d78-748c898d057f None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:12.012914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746e1fe-5dc5-423d-ba97-3d5ff4334ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:12.013185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746e1fe-5dc5-423d-ba97-3d5ff4334ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:12.013449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746e1fe-5dc5-423d-ba97-3d5ff4334ef2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:12.023914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746e1fe-5dc5-423d-ba97-3d5ff4334ef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:12.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-d746e1fe-5dc5-423d-ba97-3d5ff4334ef2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:26:12.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:12.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:12.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:12.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:12.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:12.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:12.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:12.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:12.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:12.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:12.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:12.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:12.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:12.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:12.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:12.030844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:12.030844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:12.030844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:12.030844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:12.030844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:12.030844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:12.030844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:12.030844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:12.030844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:12.030844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:12.030844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:12.030844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:12.030844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:12.030844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:12.030844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:12.030844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:12.035799 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:12.035799 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:12.035799 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:12.035799 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:12.035799 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:12.035799 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:12.035799 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:12.035799 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:12.035799 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:12.035799 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:12.035799 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:26:12.035799 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:12.035799 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 79/170] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:26:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:12.035799 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746e1fe-5dc5-423d-ba97-3d5ff4334ef2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:12.054345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7826b4-b18f-41ce-86f7-b28d20266b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:12.055165 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7826b4-b18f-41ce-86f7-b28d20266b0b None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:12.055445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7826b4-b18f-41ce-86f7-b28d20266b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:12.055712 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7826b4-b18f-41ce-86f7-b28d20266b0b None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:12.056365 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7826b4-b18f-41ce-86f7-b28d20266b0b None admin] RBAC: Adding query filter params (name=member) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:12.056659 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7826b4-b18f-41ce-86f7-b28d20266b0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:12.058626 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7826b4-b18f-41ce-86f7-b28d20266b0b None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:12.066339 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 92/171] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:26:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:12.067104 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7826b4-b18f-41ce-86f7-b28d20266b0b None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:12.134171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746e1fe-5dc5-423d-ba97-3d5ff4334ef2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:12.164511 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7826b4-b18f-41ce-86f7-b28d20266b0b None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:12.167664 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7243d-b671-465c-ae4d-c55994ea5cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:12.168258 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7243d-b671-465c-ae4d-c55994ea5cc2 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:12.168531 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7243d-b671-465c-ae4d-c55994ea5cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:12.168796 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7243d-b671-465c-ae4d-c55994ea5cc2 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:12.169433 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c7243d-b671-465c-ae4d-c55994ea5cc2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:12.170861 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c7243d-b671-465c-ae4d-c55994ea5cc2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:12.192652 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 80/172] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:26:12 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:26:12.207347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7243d-b671-465c-ae4d-c55994ea5cc2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:12.207467 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa08762-660d-4bde-ae63-023c9f07e8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:12.207467 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa08762-660d-4bde-ae63-023c9f07e8bc None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:12.207467 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa08762-660d-4bde-ae63-023c9f07e8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:12.207467 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa08762-660d-4bde-ae63-023c9f07e8bc None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:12.207801 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa08762-660d-4bde-ae63-023c9f07e8bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:12.207801 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa08762-660d-4bde-ae63-023c9f07e8bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:12.207801 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa08762-660d-4bde-ae63-023c9f07e8bc None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:12.215884 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 93/173] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:12.216757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa08762-660d-4bde-ae63-023c9f07e8bc None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:12.235697 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:12.263635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:12.327834 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7243d-b671-465c-ae4d-c55994ea5cc2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:12.329130 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa08762-660d-4bde-ae63-023c9f07e8bc None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:12.366727 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3710d8-5ce3-416a-91c5-5309e76a91ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:12.366727 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3710d8-5ce3-416a-91c5-5309e76a91ea None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:12.366727 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3710d8-5ce3-416a-91c5-5309e76a91ea None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:12.367099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3710d8-5ce3-416a-91c5-5309e76a91ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:12.367999 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d047f4-9ee8-46c5-9b7b-dcfaa745927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:12.368893 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d047f4-9ee8-46c5-9b7b-dcfaa745927c None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:12.368983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d047f4-9ee8-46c5-9b7b-dcfaa745927c None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:12.369233 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d047f4-9ee8-46c5-9b7b-dcfaa745927c None admin] PATH_INFO: `/v3/users/849a4a49089e408fb4c09462695bf66f` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:12.381021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3710d8-5ce3-416a-91c5-5309e76a91ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:12.387064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d047f4-9ee8-46c5-9b7b-dcfaa745927c None admin] RBAC: Authorizing `identity:get_user(user_id=849a4a49089e408fb4c09462695bf66f)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:12.388025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-2d3710d8-5ce3-416a-91c5-5309e76a91ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:26:12.388025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:12.388025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:12.388025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:12.388025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:12.388025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:12.388025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:12.388025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:12.388025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:12.388025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:12.388025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:12.388025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:12.388025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:12.388025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:12.388025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:12.388025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:12.388896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:12.388896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:12.388896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:12.388896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:12.388896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:12.388896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:12.388896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:12.388896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:12.388896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:12.388896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:12.388896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:12.388896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:12.388896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:12.388896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:12.388896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:12.388896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:12.389748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:12.389748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:12.389748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:12.389748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:12.389748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:12.389748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:12.389748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:12.389748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:12.389748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:12.389748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:12.389748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:26:12.389748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:12.390389 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d047f4-9ee8-46c5-9b7b-dcfaa745927c None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:12.390389 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 94/174] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:12 2022] GET /identity/v3/users/849a4a49089e408fb4c09462695bf66f => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:12.391758 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 81/175] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Oct 12 14:26:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:12.391996 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d047f4-9ee8-46c5-9b7b-dcfaa745927c None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:12.396162 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3710d8-5ce3-416a-91c5-5309e76a91ea None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:12.474365 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:12.511237 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d047f4-9ee8-46c5-9b7b-dcfaa745927c None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:12.528963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3710d8-5ce3-416a-91c5-5309e76a91ea None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:12.532893 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:12.553710 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6eda91-9fa9-4a1a-bdde-f88790cde271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:12.554339 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6eda91-9fa9-4a1a-bdde-f88790cde271 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:12.554614 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6eda91-9fa9-4a1a-bdde-f88790cde271 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:12.554877 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6eda91-9fa9-4a1a-bdde-f88790cde271 None admin] PATH_INFO: `/v3/users/c013522a3a114b4ab2b59f659a080ee6` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:12.557440 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6eda91-9fa9-4a1a-bdde-f88790cde271 None admin] RBAC: Authorizing `identity:get_user(user_id=c013522a3a114b4ab2b59f659a080ee6)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:12.559065 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6eda91-9fa9-4a1a-bdde-f88790cde271 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:12.560597 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 95/176] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:12 2022] GET /identity/v3/users/c013522a3a114b4ab2b59f659a080ee6 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:12.561406 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6eda91-9fa9-4a1a-bdde-f88790cde271 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:12.589564 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9a2ab-4e45-48a3-874e-e8832c881e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:12.590492 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9a2ab-4e45-48a3-874e-e8832c881e44 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:12.590769 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9a2ab-4e45-48a3-874e-e8832c881e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:12.591471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9a2ab-4e45-48a3-874e-e8832c881e44 None admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:12.592704 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9a2ab-4e45-48a3-874e-e8832c881e44 None admin] RBAC: Authorizing `identity:get_project(project_id=a78b76de457545f7927d98a7a89f4cec)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:12.594720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9a2ab-4e45-48a3-874e-e8832c881e44 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:12.597271 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 82/177] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:12 2022] GET /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:12.598580 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9a2ab-4e45-48a3-874e-e8832c881e44 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:12.678359 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6eda91-9fa9-4a1a-bdde-f88790cde271 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:12.714658 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf3658-df67-452c-a999-44500a82d87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:12.715481 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf3658-df67-452c-a999-44500a82d87c None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:12.715848 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf3658-df67-452c-a999-44500a82d87c None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:12.716179 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf3658-df67-452c-a999-44500a82d87c None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:12.717301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf3658-df67-452c-a999-44500a82d87c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:12.717560 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf3658-df67-452c-a999-44500a82d87c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:12.719014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf3658-df67-452c-a999-44500a82d87c None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:12.725044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:12.730312 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 96/178] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:12.730951 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf3658-df67-452c-a999-44500a82d87c None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:12.733427 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9a2ab-4e45-48a3-874e-e8832c881e44 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:12.777706 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d342f3-601c-406b-9a96-ca8c082a3417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:12.779621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d342f3-601c-406b-9a96-ca8c082a3417 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:12.780133 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d342f3-601c-406b-9a96-ca8c082a3417 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:12.780777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d342f3-601c-406b-9a96-ca8c082a3417 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:12.832442 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:12.848929 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf3658-df67-452c-a999-44500a82d87c None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:12.863573 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d342f3-601c-406b-9a96-ca8c082a3417 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:12.865134 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d342f3-601c-406b-9a96-ca8c082a3417 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:12.902886 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447bc90-3661-4b43-8434-f509ee1c6d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:12.903570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447bc90-3661-4b43-8434-f509ee1c6d14 None admin] REQUEST_METHOD: `PUT` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:12.903843 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447bc90-3661-4b43-8434-f509ee1c6d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:12.904111 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447bc90-3661-4b43-8434-f509ee1c6d14 None admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec/users/849a4a49089e408fb4c09462695bf66f/roles/6d1894444bfd400a89591929bd8925de` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:12.909058 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447bc90-3661-4b43-8434-f509ee1c6d14 None admin] RBAC: Authorizing `identity:create_grant(project_id=a78b76de457545f7927d98a7a89f4cec, user_id=849a4a49089e408fb4c09462695bf66f, role_id=6d1894444bfd400a89591929bd8925de)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:12.910795 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447bc90-3661-4b43-8434-f509ee1c6d14 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:12.929276 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 97/179] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:26:12 2022] PUT /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec/users/849a4a49089e408fb4c09462695bf66f/roles/6d1894444bfd400a89591929bd8925de => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:26:12.930685 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447bc90-3661-4b43-8434-f509ee1c6d14 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:13.016756 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 83/180] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 14:26:13.023367 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d342f3-601c-406b-9a96-ca8c082a3417 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:13.056536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:13.134674 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447bc90-3661-4b43-8434-f509ee1c6d14 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:13.165659 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:13.191751 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a2bab-66ad-4fa1-8298-4b5c19e0e743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:13.192910 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a2bab-66ad-4fa1-8298-4b5c19e0e743 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:13.193406 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a2bab-66ad-4fa1-8298-4b5c19e0e743 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:13.194046 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a2bab-66ad-4fa1-8298-4b5c19e0e743 None admin] PATH_INFO: `/v3/users/c013522a3a114b4ab2b59f659a080ee6` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:13.196142 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a2bab-66ad-4fa1-8298-4b5c19e0e743 None admin] RBAC: Authorizing `identity:get_user(user_id=c013522a3a114b4ab2b59f659a080ee6)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:13.198072 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a2bab-66ad-4fa1-8298-4b5c19e0e743 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:13.200679 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 98/181] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:12 2022] GET /identity/v3/users/c013522a3a114b4ab2b59f659a080ee6 => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:13.207243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a2bab-66ad-4fa1-8298-4b5c19e0e743 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:13.209887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d342f3-601c-406b-9a96-ca8c082a3417 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:13.251357 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02201619-b3f3-4e2d-8e97-730fd0d10691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:13.252083 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02201619-b3f3-4e2d-8e97-730fd0d10691 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:13.252429 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02201619-b3f3-4e2d-8e97-730fd0d10691 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:13.252786 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02201619-b3f3-4e2d-8e97-730fd0d10691 None admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:13.253983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02201619-b3f3-4e2d-8e97-730fd0d10691 None admin] RBAC: Authorizing `identity:get_project(project_id=a78b76de457545f7927d98a7a89f4cec)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:13.256130 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02201619-b3f3-4e2d-8e97-730fd0d10691 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:13.262838 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 84/182] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:13 2022] GET /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:13.264735 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02201619-b3f3-4e2d-8e97-730fd0d10691 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:13.368537 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a2bab-66ad-4fa1-8298-4b5c19e0e743 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:13.382487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02201619-b3f3-4e2d-8e97-730fd0d10691 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:13.419510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:13.433320 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4c30c-5701-4a06-bfb2-a0726019e30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:13.433963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4c30c-5701-4a06-bfb2-a0726019e30d None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:13.434239 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4c30c-5701-4a06-bfb2-a0726019e30d None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:13.434506 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4c30c-5701-4a06-bfb2-a0726019e30d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:13.449900 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544a9a3-7462-4cd5-a9f4-15f25b91f809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:13.450445 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544a9a3-7462-4cd5-a9f4-15f25b91f809 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:13.450720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544a9a3-7462-4cd5-a9f4-15f25b91f809 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:13.450991 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544a9a3-7462-4cd5-a9f4-15f25b91f809 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:13.451746 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544a9a3-7462-4cd5-a9f4-15f25b91f809 None admin] RBAC: Adding query filter params (role.id=4b1f48ab911e4fadbc70164ae43d508b, scope.project.id=a78b76de457545f7927d98a7a89f4cec, user.id=c013522a3a114b4ab2b59f659a080ee6) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:13.452103 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544a9a3-7462-4cd5-a9f4-15f25b91f809 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:13.453608 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544a9a3-7462-4cd5-a9f4-15f25b91f809 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:13.462587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 85/183] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:26:13 2022] GET /identity/v3/role_assignments?user.id=c013522a3a114b4ab2b59f659a080ee6&scope.project.id=a78b76de457545f7927d98a7a89f4cec&role.id=4b1f48ab911e4fadbc70164ae43d508b => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:13.566799 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4c30c-5701-4a06-bfb2-a0726019e30d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:13.568644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4c30c-5701-4a06-bfb2-a0726019e30d None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:13.696338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 99/184] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 14:26:13.703094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544a9a3-7462-4cd5-a9f4-15f25b91f809 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:13.707057 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:13.845914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544a9a3-7462-4cd5-a9f4-15f25b91f809 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:13.881887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b95263-f147-4291-a677-054c917ee860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:13.882526 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b95263-f147-4291-a677-054c917ee860 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:13.882801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b95263-f147-4291-a677-054c917ee860 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:13.883068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b95263-f147-4291-a677-054c917ee860 None admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:13.884222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b95263-f147-4291-a677-054c917ee860 None admin] RBAC: Authorizing `identity:get_project(project_id=a78b76de457545f7927d98a7a89f4cec)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:13.886187 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b95263-f147-4291-a677-054c917ee860 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:13.892729 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 86/185] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:13 2022] GET /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:13.894925 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4c30c-5701-4a06-bfb2-a0726019e30d None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:13.929439 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:14.001807 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4c30c-5701-4a06-bfb2-a0726019e30d None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:14.039954 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13306a84-9fea-4e33-9942-cb3cb7461bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:14.040584 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13306a84-9fea-4e33-9942-cb3cb7461bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:14.040859 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13306a84-9fea-4e33-9942-cb3cb7461bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:14.041838 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13306a84-9fea-4e33-9942-cb3cb7461bd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:14.042708 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13306a84-9fea-4e33-9942-cb3cb7461bd1 None admin] RBAC: Adding query filter params (role.id=f0761dd38b794ee6bfab00ac01f240cd, scope.project.id=a78b76de457545f7927d98a7a89f4cec, user.id=c013522a3a114b4ab2b59f659a080ee6) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:14.043021 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13306a84-9fea-4e33-9942-cb3cb7461bd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:14.044555 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13306a84-9fea-4e33-9942-cb3cb7461bd1 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:14.051475 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 100/186] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:26:13 2022] GET /identity/v3/role_assignments?user.id=c013522a3a114b4ab2b59f659a080ee6&scope.project.id=a78b76de457545f7927d98a7a89f4cec&role.id=f0761dd38b794ee6bfab00ac01f240cd => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:14.578054 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bdecb-19ea-4ee6-8c29-dfff634ef67b None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:14.578585 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bdecb-19ea-4ee6-8c29-dfff634ef67b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:14.578585 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bdecb-19ea-4ee6-8c29-dfff634ef67b None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:14.580027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 87/187] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:14.585124 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe2b42-3e9a-4079-a1b6-497dbde9165e None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:14.585881 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe2b42-3e9a-4079-a1b6-497dbde9165e None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:14.586155 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe2b42-3e9a-4079-a1b6-497dbde9165e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:14.605567 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-8abe2b42-3e9a-4079-a1b6-497dbde9165e None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:14.606740 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-8abe2b42-3e9a-4079-a1b6-497dbde9165e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:14.674879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 101/188] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:14.823956 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c63c4e-6f36-4bbe-a110-ac5bab480b1f None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:14.824339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c63c4e-6f36-4bbe-a110-ac5bab480b1f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:14.824683 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c63c4e-6f36-4bbe-a110-ac5bab480b1f None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:14.826122 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 88/189] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:14.831446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b4998-d770-46e9-becf-bb23b1c4f0f5 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:14.831675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b4998-d770-46e9-becf-bb23b1c4f0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:14.831944 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b4998-d770-46e9-becf-bb23b1c4f0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:14.849811 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-1e9b4998-d770-46e9-becf-bb23b1c4f0f5 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:14.850967 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-1e9b4998-d770-46e9-becf-bb23b1c4f0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:14.868369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:14.920867 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 102/190] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:14.943533 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c63c4e-6f36-4bbe-a110-ac5bab480b1f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:15.016656 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:15.050319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c63c4e-6f36-4bbe-a110-ac5bab480b1f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:15.082790 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a844fce-1049-4fa4-83ce-7721f3ec5c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:15.083374 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a844fce-1049-4fa4-83ce-7721f3ec5c58 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:15.083653 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a844fce-1049-4fa4-83ce-7721f3ec5c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:15.083920 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a844fce-1049-4fa4-83ce-7721f3ec5c58 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:15.094551 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a844fce-1049-4fa4-83ce-7721f3ec5c58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:15.099876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-3a844fce-1049-4fa4-83ce-7721f3ec5c58 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 14:26:15.099876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:15.099876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:15.099876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:15.099876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:15.099876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:15.099876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:15.099876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:15.099876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:15.099876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:15.099876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:15.099876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:15.099876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:15.099876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:15.099876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:15.099876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:15.100733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:15.100733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:15.100733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:15.100733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:15.100733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:15.100733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:15.100733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:15.100733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:15.100733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:15.100733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:15.100733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:15.100733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:15.100733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:15.100733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:15.100733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:15.100733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:15.101521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:15.101521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:15.101521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:15.101521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:15.101521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:15.101521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:15.101521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:15.101521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:15.101521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:15.101521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:15.101521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 14:26:15.101521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:15.101521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 89/191] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:15.108651 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b4998-d770-46e9-becf-bb23b1c4f0f5 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:15.192385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a844fce-1049-4fa4-83ce-7721f3ec5c58 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:15.212605 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b4998-d770-46e9-becf-bb23b1c4f0f5 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:15.255285 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333801ef-d398-436d-a42c-be9efecdfef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:15.256324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333801ef-d398-436d-a42c-be9efecdfef7 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:15.256601 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333801ef-d398-436d-a42c-be9efecdfef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:15.257143 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333801ef-d398-436d-a42c-be9efecdfef7 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:15.257842 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333801ef-d398-436d-a42c-be9efecdfef7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:15.258145 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333801ef-d398-436d-a42c-be9efecdfef7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:15.260432 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333801ef-d398-436d-a42c-be9efecdfef7 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:15.271535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 103/192] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:26:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:15.278287 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333801ef-d398-436d-a42c-be9efecdfef7 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:15.297372 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:15.312320 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a844fce-1049-4fa4-83ce-7721f3ec5c58 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:15.350540 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:15.357411 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ed8c1-4d33-4800-877d-cc6de1f7b21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:15.358230 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ed8c1-4d33-4800-877d-cc6de1f7b21d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:15.358512 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ed8c1-4d33-4800-877d-cc6de1f7b21d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:15.358779 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ed8c1-4d33-4800-877d-cc6de1f7b21d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:15.361005 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ed8c1-4d33-4800-877d-cc6de1f7b21d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:15.362466 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ed8c1-4d33-4800-877d-cc6de1f7b21d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:15.364315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 90/193] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:15.370528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ed8c1-4d33-4800-877d-cc6de1f7b21d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:15.396530 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333801ef-d398-436d-a42c-be9efecdfef7 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:15.431006 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45554be6-99d3-4b11-a558-9f1d70b26433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:15.432528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45554be6-99d3-4b11-a558-9f1d70b26433 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:15.432864 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45554be6-99d3-4b11-a558-9f1d70b26433 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:15.433132 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45554be6-99d3-4b11-a558-9f1d70b26433 None admin] PATH_INFO: `/v3/users/4df333fde0ff4b29bd8419a088bb0c61` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:15.448560 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45554be6-99d3-4b11-a558-9f1d70b26433 None admin] RBAC: Authorizing `identity:get_user(user_id=4df333fde0ff4b29bd8419a088bb0c61)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:15.450020 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45554be6-99d3-4b11-a558-9f1d70b26433 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:15.457861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 104/194] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:15 2022] GET /identity/v3/users/4df333fde0ff4b29bd8419a088bb0c61 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:15.457861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f59686-2359-4732-8560-cd6c3c4a306c None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:15.457861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f59686-2359-4732-8560-cd6c3c4a306c None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:15.457861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f59686-2359-4732-8560-cd6c3c4a306c None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:15.457861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 105/195] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:15.458815 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f59686-2359-4732-8560-cd6c3c4a306c None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:15.490675 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ed8c1-4d33-4800-877d-cc6de1f7b21d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:15.525348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff0e96-23e9-4e96-a0f7-bd1b949edd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:15.526208 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff0e96-23e9-4e96-a0f7-bd1b949edd7b None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:15.526483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff0e96-23e9-4e96-a0f7-bd1b949edd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:15.526749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff0e96-23e9-4e96-a0f7-bd1b949edd7b None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:15.527442 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff0e96-23e9-4e96-a0f7-bd1b949edd7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:15.528882 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff0e96-23e9-4e96-a0f7-bd1b949edd7b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:15.536861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:15.552841 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 91/196] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:26:15 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 14:26:15.554087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284f930-1b60-4742-8df4-fabb6b26eeb9 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:15.554366 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284f930-1b60-4742-8df4-fabb6b26eeb9 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:15.554638 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284f930-1b60-4742-8df4-fabb6b26eeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:15.577926 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c284f930-1b60-4742-8df4-fabb6b26eeb9 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:15.579242 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c284f930-1b60-4742-8df4-fabb6b26eeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:15.592511 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f59686-2359-4732-8560-cd6c3c4a306c None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:15.639210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aaaabb-f049-4241-9ff5-2a5ba6756a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:15.639803 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aaaabb-f049-4241-9ff5-2a5ba6756a2c None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:15.640070 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aaaabb-f049-4241-9ff5-2a5ba6756a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:15.640433 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aaaabb-f049-4241-9ff5-2a5ba6756a2c None admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:15.641476 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aaaabb-f049-4241-9ff5-2a5ba6756a2c None admin] RBAC: Authorizing `identity:get_project(project_id=a78b76de457545f7927d98a7a89f4cec)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:15.643044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aaaabb-f049-4241-9ff5-2a5ba6756a2c None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:15.643654 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:15.644843 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 106/197] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:15 2022] GET /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:15.652416 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aaaabb-f049-4241-9ff5-2a5ba6756a2c None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:15.668227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 92/198] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:15.769690 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aaaabb-f049-4241-9ff5-2a5ba6756a2c None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:15.812987 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b077c-11b4-49b0-903e-9d13681a19aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:15.814000 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b077c-11b4-49b0-903e-9d13681a19aa None admin] REQUEST_METHOD: `PUT` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:15.814446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b077c-11b4-49b0-903e-9d13681a19aa None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:15.814878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b077c-11b4-49b0-903e-9d13681a19aa None admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec/users/4df333fde0ff4b29bd8419a088bb0c61/roles/ac952ec5ebb24b2fa6476a7990e40147` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:15.817089 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b077c-11b4-49b0-903e-9d13681a19aa None admin] RBAC: Authorizing `identity:create_grant(project_id=a78b76de457545f7927d98a7a89f4cec, user_id=4df333fde0ff4b29bd8419a088bb0c61, role_id=ac952ec5ebb24b2fa6476a7990e40147)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:15.818613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b077c-11b4-49b0-903e-9d13681a19aa None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:15.830659 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:15.835510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 107/199] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:26:15 2022] PUT /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec/users/4df333fde0ff4b29bd8419a088bb0c61/roles/ac952ec5ebb24b2fa6476a7990e40147 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 14:26:15.971969 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284f930-1b60-4742-8df4-fabb6b26eeb9 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:16.109208 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284f930-1b60-4742-8df4-fabb6b26eeb9 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:16.155479 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61009250-efe5-44bd-a1f3-62cf22705915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:16.156575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61009250-efe5-44bd-a1f3-62cf22705915 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:16.157055 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61009250-efe5-44bd-a1f3-62cf22705915 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:16.157547 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61009250-efe5-44bd-a1f3-62cf22705915 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:16.171013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61009250-efe5-44bd-a1f3-62cf22705915 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:16.176477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-61009250-efe5-44bd-a1f3-62cf22705915 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:16.176477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:16.176477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:16.176477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:16.176477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:16.176477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:16.176477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:16.176477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:16.176477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:16.176477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:16.176477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:16.176477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:16.176477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:16.176477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:16.176477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:16.176477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:16.177522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:16.177522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:16.177522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:16.177522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:16.177522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:16.177522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:16.177522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:16.177522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:16.177522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:16.177522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:16.177522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:16.177522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:16.177522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:16.177522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:16.177522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:16.177522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:16.178487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:16.178487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:16.178487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:16.178487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:16.178487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:16.178487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:16.178487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:16.178487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:16.178487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:16.178487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:16.178487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:16.178487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:16.180510 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 93/200] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:16.187957 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b077c-11b4-49b0-903e-9d13681a19aa None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:16.310783 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b077c-11b4-49b0-903e-9d13681a19aa None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:16.347048 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852619a9-ca0f-46c2-8307-de0243007dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:16.347925 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852619a9-ca0f-46c2-8307-de0243007dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:16.348346 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852619a9-ca0f-46c2-8307-de0243007dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:16.348784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852619a9-ca0f-46c2-8307-de0243007dd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:16.349584 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852619a9-ca0f-46c2-8307-de0243007dd6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:16.350025 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852619a9-ca0f-46c2-8307-de0243007dd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:16.351580 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852619a9-ca0f-46c2-8307-de0243007dd6 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:16.361218 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 108/201] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:26:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:16.368433 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61009250-efe5-44bd-a1f3-62cf22705915 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:16.418934 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:16.475333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61009250-efe5-44bd-a1f3-62cf22705915 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:16.511178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4259c-41c3-4878-836c-4efb1eccbeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:16.511836 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4259c-41c3-4878-836c-4efb1eccbeb3 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:16.512114 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4259c-41c3-4878-836c-4efb1eccbeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:16.512381 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4259c-41c3-4878-836c-4efb1eccbeb3 None admin] PATH_INFO: `/v3/users/849a4a49089e408fb4c09462695bf66f` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:16.513540 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4259c-41c3-4878-836c-4efb1eccbeb3 None admin] RBAC: Authorizing `identity:get_user(user_id=849a4a49089e408fb4c09462695bf66f)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:16.514992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4259c-41c3-4878-836c-4efb1eccbeb3 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:16.516903 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 94/202] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:16 2022] GET /identity/v3/users/849a4a49089e408fb4c09462695bf66f => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:16.522505 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852619a9-ca0f-46c2-8307-de0243007dd6 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:16.590258 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:16.633689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852619a9-ca0f-46c2-8307-de0243007dd6 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:16.670333 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629a2f0-38ed-42e7-992b-84e4fab2456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:16.670959 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629a2f0-38ed-42e7-992b-84e4fab2456b None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:16.671273 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629a2f0-38ed-42e7-992b-84e4fab2456b None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:16.671548 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629a2f0-38ed-42e7-992b-84e4fab2456b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:16.745863 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629a2f0-38ed-42e7-992b-84e4fab2456b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:16.747377 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629a2f0-38ed-42e7-992b-84e4fab2456b None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:16.813490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:16.857051 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 109/203] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 12 14:26:16.862710 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4259c-41c3-4878-836c-4efb1eccbeb3 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:16.964307 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4259c-41c3-4878-836c-4efb1eccbeb3 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:16.978823 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:16.999224 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f92831-9533-4b83-a5dc-8961fa101df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:17.000086 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f92831-9533-4b83-a5dc-8961fa101df6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:17.000366 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f92831-9533-4b83-a5dc-8961fa101df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:17.000642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f92831-9533-4b83-a5dc-8961fa101df6 None admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:17.001776 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f92831-9533-4b83-a5dc-8961fa101df6 None admin] RBAC: Authorizing `identity:get_project(project_id=a78b76de457545f7927d98a7a89f4cec)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:17.003275 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f92831-9533-4b83-a5dc-8961fa101df6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:17.005204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 95/204] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:16 2022] GET /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:17.011070 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629a2f0-38ed-42e7-992b-84e4fab2456b None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:17.112981 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629a2f0-38ed-42e7-992b-84e4fab2456b None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:17.147603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c1046-6615-42c5-9476-8d5fa9537303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:17.148202 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c1046-6615-42c5-9476-8d5fa9537303 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:17.148474 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c1046-6615-42c5-9476-8d5fa9537303 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:17.148741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c1046-6615-42c5-9476-8d5fa9537303 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:17.149458 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c1046-6615-42c5-9476-8d5fa9537303 None admin] RBAC: Adding query filter params (role.id=6d1894444bfd400a89591929bd8925de, scope.project.id=a78b76de457545f7927d98a7a89f4cec, user.id=849a4a49089e408fb4c09462695bf66f) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:17.149814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c1046-6615-42c5-9476-8d5fa9537303 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:17.151725 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c1046-6615-42c5-9476-8d5fa9537303 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:17.161740 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 110/205] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:26:17 2022] GET /identity/v3/role_assignments?user.id=849a4a49089e408fb4c09462695bf66f&scope.project.id=a78b76de457545f7927d98a7a89f4cec&role.id=6d1894444bfd400a89591929bd8925de => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:18.015964 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d25fb-3215-417b-a63b-a7f778bd67c4 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:18.016516 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d25fb-3215-417b-a63b-a7f778bd67c4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:18.016516 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d25fb-3215-417b-a63b-a7f778bd67c4 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:18.017920 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 96/206] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:18.024180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e58f84-ef6b-4a94-b749-b0235792b651 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:18.025169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e58f84-ef6b-4a94-b749-b0235792b651 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:18.025243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e58f84-ef6b-4a94-b749-b0235792b651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:18.044671 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-a3e58f84-ef6b-4a94-b749-b0235792b651 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:18.045848 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-a3e58f84-ef6b-4a94-b749-b0235792b651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:18.067495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:18.114954 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d13217-7a3a-43a3-9bcf-369ce4527ab9 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:18.115262 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d13217-7a3a-43a3-9bcf-369ce4527ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:18.115705 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d13217-7a3a-43a3-9bcf-369ce4527ab9 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:18.117036 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 97/207] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:18.117705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 111/208] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:26:18.123837 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34db5b9-7b07-4414-9100-6c96b0052c88 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:18.124191 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34db5b9-7b07-4414-9100-6c96b0052c88 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:18.124866 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34db5b9-7b07-4414-9100-6c96b0052c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:18.125884 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d749f-0db1-4f2d-aec3-4ae47d975486 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:18.126161 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d749f-0db1-4f2d-aec3-4ae47d975486 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:18.126426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d749f-0db1-4f2d-aec3-4ae47d975486 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:18.127958 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 112/209] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:26:18.133157 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d11926-fbbf-4325-b531-8b9aeb3da7bf None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:18.135516 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d11926-fbbf-4325-b531-8b9aeb3da7bf None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:18.135800 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d11926-fbbf-4325-b531-8b9aeb3da7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:18.144182 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d34db5b9-7b07-4414-9100-6c96b0052c88 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:18.146368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d34db5b9-7b07-4414-9100-6c96b0052c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:18.157345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-c7d11926-fbbf-4325-b531-8b9aeb3da7bf None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:18.158337 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-c7d11926-fbbf-4325-b531-8b9aeb3da7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:18.219537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 98/210] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:26:18.235734 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 113/211] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:18.390236 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34db5b9-7b07-4414-9100-6c96b0052c88 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:18.489111 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d11926-fbbf-4325-b531-8b9aeb3da7bf None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:18.491258 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34db5b9-7b07-4414-9100-6c96b0052c88 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:18.505832 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:18.527031 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1904add-4f10-4419-9dad-8751ef15c0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:18.527687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1904add-4f10-4419-9dad-8751ef15c0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:18.527964 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1904add-4f10-4419-9dad-8751ef15c0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:18.528232 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1904add-4f10-4419-9dad-8751ef15c0c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:18.538037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1904add-4f10-4419-9dad-8751ef15c0c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:18.543500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-f1904add-4f10-4419-9dad-8751ef15c0c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:18.543500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:18.543500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:18.543500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:18.543500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:18.543500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:18.543500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:18.543500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:18.543500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:18.543500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:18.543500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:18.543500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:18.543500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:18.543500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:18.543500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:18.543500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:18.544452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:18.544452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:18.544452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:18.544452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:18.544452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:18.544452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:18.544452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:18.544452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:18.544452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:18.544452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:18.544452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:18.544452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:18.544452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:18.544452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:18.544452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:18.544452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:18.545527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:18.545527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:18.545527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:18.545527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:18.545527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:18.545527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:18.545527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:18.545527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:18.545527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:18.545527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:18.545527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:18.545527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:18.545527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 99/212] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:18.550065 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1904add-4f10-4419-9dad-8751ef15c0c4 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:18.576707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:18.593018 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d11926-fbbf-4325-b531-8b9aeb3da7bf None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:18.626621 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26400696-8ad3-43d2-a5f5-d3792efd400c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:18.628508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26400696-8ad3-43d2-a5f5-d3792efd400c None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:18.628849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26400696-8ad3-43d2-a5f5-d3792efd400c None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:18.629360 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26400696-8ad3-43d2-a5f5-d3792efd400c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:18.630248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26400696-8ad3-43d2-a5f5-d3792efd400c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:18.631866 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26400696-8ad3-43d2-a5f5-d3792efd400c None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:18.633613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 114/213] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:18.635350 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26400696-8ad3-43d2-a5f5-d3792efd400c None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:18.652085 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1904add-4f10-4419-9dad-8751ef15c0c4 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:18.685238 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7ac73-fe1e-4196-aaa9-91f7d6700ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:18.685859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7ac73-fe1e-4196-aaa9-91f7d6700ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:18.686136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7ac73-fe1e-4196-aaa9-91f7d6700ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:18.686408 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7ac73-fe1e-4196-aaa9-91f7d6700ce3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:18.697320 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7ac73-fe1e-4196-aaa9-91f7d6700ce3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:18.702190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-56a7ac73-fe1e-4196-aaa9-91f7d6700ce3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 14:26:18.702190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:18.702190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:18.702190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:18.702190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:18.702190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:18.702190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:18.702190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:18.702190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:18.702190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:18.702190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:18.702190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:18.702190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:18.702190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:18.702190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:18.702190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:18.703024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:18.703024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:18.703024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:18.703024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:18.703024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:18.703024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:18.703024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:18.703024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:18.703024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:18.703024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:18.703024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:18.703024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:18.703024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:18.703024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:18.703024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:18.703024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:18.703836 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:18.703836 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:18.703836 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:18.703836 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:18.703836 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:18.703836 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:18.703836 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:18.703836 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:18.703836 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:18.703836 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:18.703836 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 14:26:18.703836 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:18.703836 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 100/214] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:18.704454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7ac73-fe1e-4196-aaa9-91f7d6700ce3 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:18.736238 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:18.737189 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26400696-8ad3-43d2-a5f5-d3792efd400c None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:18.770734 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc1f4c-e04b-4a71-9a26-1bc3056ea7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:18.771360 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc1f4c-e04b-4a71-9a26-1bc3056ea7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:18.771628 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc1f4c-e04b-4a71-9a26-1bc3056ea7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:18.771886 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc1f4c-e04b-4a71-9a26-1bc3056ea7b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:18.772527 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc1f4c-e04b-4a71-9a26-1bc3056ea7b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:18.772814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc1f4c-e04b-4a71-9a26-1bc3056ea7b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:18.774168 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc1f4c-e04b-4a71-9a26-1bc3056ea7b9 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:18.782474 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 115/215] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:26:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:18.783268 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc1f4c-e04b-4a71-9a26-1bc3056ea7b9 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:18.800790 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:18.810775 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7ac73-fe1e-4196-aaa9-91f7d6700ce3 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:18.844897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4f99e-4df3-475e-8f07-e92261fc39cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:18.845500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4f99e-4df3-475e-8f07-e92261fc39cb None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:18.845775 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4f99e-4df3-475e-8f07-e92261fc39cb None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:18.846042 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4f99e-4df3-475e-8f07-e92261fc39cb None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:18.846685 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4f99e-4df3-475e-8f07-e92261fc39cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:18.848147 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4f99e-4df3-475e-8f07-e92261fc39cb None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:18.868940 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 101/216] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:26:18 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:26:18.869759 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4f99e-4df3-475e-8f07-e92261fc39cb None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:18.887800 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc1f4c-e04b-4a71-9a26-1bc3056ea7b9 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:18.925816 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5d8fc-609c-4b30-9827-2822df157da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:18.926446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5d8fc-609c-4b30-9827-2822df157da9 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:18.926724 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5d8fc-609c-4b30-9827-2822df157da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:18.927000 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5d8fc-609c-4b30-9827-2822df157da9 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:18.927716 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5d8fc-609c-4b30-9827-2822df157da9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:18.928018 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5d8fc-609c-4b30-9827-2822df157da9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:18.929581 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5d8fc-609c-4b30-9827-2822df157da9 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:18.943228 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 116/217] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:26:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:18.945331 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5d8fc-609c-4b30-9827-2822df157da9 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:18.982445 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:18.999192 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4f99e-4df3-475e-8f07-e92261fc39cb None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:19.034107 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:19.045463 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc3283-b6b1-45c5-a0d2-8f8336a4271f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:19.046531 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc3283-b6b1-45c5-a0d2-8f8336a4271f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:19.046808 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc3283-b6b1-45c5-a0d2-8f8336a4271f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:19.047080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc3283-b6b1-45c5-a0d2-8f8336a4271f None admin] PATH_INFO: `/v3/users/1545576dc9de4dedb41dd5712d682a09` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:19.062293 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5d8fc-609c-4b30-9827-2822df157da9 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:19.064362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc3283-b6b1-45c5-a0d2-8f8336a4271f None admin] RBAC: Authorizing `identity:get_user(user_id=1545576dc9de4dedb41dd5712d682a09)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:19.065969 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc3283-b6b1-45c5-a0d2-8f8336a4271f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:19.067939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 102/218] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:18 2022] GET /identity/v3/users/1545576dc9de4dedb41dd5712d682a09 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:19.073840 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc3283-b6b1-45c5-a0d2-8f8336a4271f None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:19.100743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e269bf-9b9b-4303-bd68-5477e4ea8f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:19.101378 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e269bf-9b9b-4303-bd68-5477e4ea8f43 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:19.101648 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e269bf-9b9b-4303-bd68-5477e4ea8f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:19.101915 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e269bf-9b9b-4303-bd68-5477e4ea8f43 None admin] PATH_INFO: `/v3/users/4df333fde0ff4b29bd8419a088bb0c61` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:19.103111 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e269bf-9b9b-4303-bd68-5477e4ea8f43 None admin] RBAC: Authorizing `identity:get_user(user_id=4df333fde0ff4b29bd8419a088bb0c61)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:19.104603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e269bf-9b9b-4303-bd68-5477e4ea8f43 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:19.106194 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 117/219] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:18 2022] GET /identity/v3/users/4df333fde0ff4b29bd8419a088bb0c61 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:19.111716 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e269bf-9b9b-4303-bd68-5477e4ea8f43 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:19.201942 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc3283-b6b1-45c5-a0d2-8f8336a4271f None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:19.237038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e269bf-9b9b-4303-bd68-5477e4ea8f43 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:19.247283 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21760c-726b-4d2f-a507-526bd18616ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:19.248275 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21760c-726b-4d2f-a507-526bd18616ef None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:19.248553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21760c-726b-4d2f-a507-526bd18616ef None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:19.248821 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21760c-726b-4d2f-a507-526bd18616ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:19.249923 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21760c-726b-4d2f-a507-526bd18616ef None admin] RBAC: Adding query filter params (role.id=6d1894444bfd400a89591929bd8925de, user.id=1545576dc9de4dedb41dd5712d682a09, scope.system=all) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:19.250244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21760c-726b-4d2f-a507-526bd18616ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:19.251994 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21760c-726b-4d2f-a507-526bd18616ef None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:19.257272 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 103/220] 10.222.0.16 () {56 vars in 1308 bytes} [Wed Oct 12 14:26:19 2022] GET /identity/v3/role_assignments?user.id=1545576dc9de4dedb41dd5712d682a09&scope.system=all&role.id=6d1894444bfd400a89591929bd8925de => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:19.279841 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d916a9-7c4e-45c8-a4ac-129066ea6f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:19.280726 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d916a9-7c4e-45c8-a4ac-129066ea6f9d None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:19.281172 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d916a9-7c4e-45c8-a4ac-129066ea6f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:19.281585 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d916a9-7c4e-45c8-a4ac-129066ea6f9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:19.300308 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:19.363850 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d916a9-7c4e-45c8-a4ac-129066ea6f9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:19.365594 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d916a9-7c4e-45c8-a4ac-129066ea6f9d None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:19.485477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 118/221] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 12 14:26:19.491280 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21760c-726b-4d2f-a507-526bd18616ef None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:19.503967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:19.602931 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21760c-726b-4d2f-a507-526bd18616ef None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:19.638644 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ccf00-455e-415f-8794-0a5d20004cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:19.639325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ccf00-455e-415f-8794-0a5d20004cba None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:19.639599 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ccf00-455e-415f-8794-0a5d20004cba None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:19.639868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ccf00-455e-415f-8794-0a5d20004cba None admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:19.640996 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ccf00-455e-415f-8794-0a5d20004cba None admin] RBAC: Authorizing `identity:get_project(project_id=a78b76de457545f7927d98a7a89f4cec)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:19.642490 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ccf00-455e-415f-8794-0a5d20004cba None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:19.644457 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 104/222] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:19 2022] GET /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:19.651020 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d916a9-7c4e-45c8-a4ac-129066ea6f9d None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:19.708147 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:19.766598 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d916a9-7c4e-45c8-a4ac-129066ea6f9d None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:19.803827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da5bcd-e47e-4eb8-a753-e0b7bf68146a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:19.804428 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da5bcd-e47e-4eb8-a753-e0b7bf68146a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:19.804697 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da5bcd-e47e-4eb8-a753-e0b7bf68146a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:19.804959 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da5bcd-e47e-4eb8-a753-e0b7bf68146a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:19.805765 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da5bcd-e47e-4eb8-a753-e0b7bf68146a None admin] RBAC: Adding query filter params (role.id=ac952ec5ebb24b2fa6476a7990e40147, scope.project.id=a78b76de457545f7927d98a7a89f4cec, user.id=4df333fde0ff4b29bd8419a088bb0c61) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:19.806067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da5bcd-e47e-4eb8-a753-e0b7bf68146a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:19.807663 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da5bcd-e47e-4eb8-a753-e0b7bf68146a None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:19.814715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 119/223] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:26:19 2022] GET /identity/v3/role_assignments?user.id=4df333fde0ff4b29bd8419a088bb0c61&scope.project.id=a78b76de457545f7927d98a7a89f4cec&role.id=ac952ec5ebb24b2fa6476a7990e40147 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:21.183643 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ca7c0-417c-418c-b790-dcf4f3ad3ba8 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:21.184196 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ca7c0-417c-418c-b790-dcf4f3ad3ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:21.184196 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ca7c0-417c-418c-b790-dcf4f3ad3ba8 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:21.187185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 105/224] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:21.191200 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc02094-fed6-4b48-bcd4-dfda7a27d44c None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:21.191622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc02094-fed6-4b48-bcd4-dfda7a27d44c None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:21.191783 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc02094-fed6-4b48-bcd4-dfda7a27d44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:21.210674 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-fdc02094-fed6-4b48-bcd4-dfda7a27d44c None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:21.212077 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-fdc02094-fed6-4b48-bcd4-dfda7a27d44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:21.255203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43d878-5850-48d9-909d-0188ae5da2ca None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:21.255494 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43d878-5850-48d9-909d-0188ae5da2ca None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:21.255756 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43d878-5850-48d9-909d-0188ae5da2ca None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:21.257097 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 106/225] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:26:21.262822 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c39a94-82cd-4165-82fd-185e5e1a9b3a None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:21.263106 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c39a94-82cd-4165-82fd-185e5e1a9b3a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:21.263411 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c39a94-82cd-4165-82fd-185e5e1a9b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:21.281428 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-06c39a94-82cd-4165-82fd-185e5e1a9b3a None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:21.282048 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 120/226] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:21.282695 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-06c39a94-82cd-4165-82fd-185e5e1a9b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:21.351030 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 107/227] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:21.461434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da351d1a-2ae6-487e-b3f6-7f1d44ed380b None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:21.461799 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da351d1a-2ae6-487e-b3f6-7f1d44ed380b None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:21.461953 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da351d1a-2ae6-487e-b3f6-7f1d44ed380b None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:21.463671 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 121/228] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:21.468598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e44e8c-60a1-4533-8b6c-dac3c723b76c None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:21.468884 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e44e8c-60a1-4533-8b6c-dac3c723b76c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:21.469155 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e44e8c-60a1-4533-8b6c-dac3c723b76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:21.487743 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a2e44e8c-60a1-4533-8b6c-dac3c723b76c None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:21.488922 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a2e44e8c-60a1-4533-8b6c-dac3c723b76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:21.490835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:21.551772 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da351d1a-2ae6-487e-b3f6-7f1d44ed380b None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:21.563304 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 108/229] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:21.602698 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:21.618338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e44e8c-60a1-4533-8b6c-dac3c723b76c None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:21.667079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da351d1a-2ae6-487e-b3f6-7f1d44ed380b None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:21.703459 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88e5c9-71aa-4e78-a8aa-b09df076d5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:21.704342 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88e5c9-71aa-4e78-a8aa-b09df076d5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:21.704816 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88e5c9-71aa-4e78-a8aa-b09df076d5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:21.705256 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88e5c9-71aa-4e78-a8aa-b09df076d5e6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:21.716359 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88e5c9-71aa-4e78-a8aa-b09df076d5e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:21.721565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-ba88e5c9-71aa-4e78-a8aa-b09df076d5e6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 14:26:21.721565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:21.721565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:21.721565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:21.721565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:21.721565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:21.721565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:21.721565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:21.721565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:21.721565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:21.721565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:21.721565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:21.721565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:21.721565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:21.721565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:21.721565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:21.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:21.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:21.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:21.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:21.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:21.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:21.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:21.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:21.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:21.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:21.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:21.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:21.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:21.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:21.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:21.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:21.723449 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:21.723449 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:21.723449 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:21.723449 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:21.723449 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:21.723449 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:21.723449 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:21.723449 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:21.723449 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:21.723449 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:21.723449 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 14:26:21.723449 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:21.729609 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 122/230] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:21.729738 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e44e8c-60a1-4533-8b6c-dac3c723b76c None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:21.730563 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88e5c9-71aa-4e78-a8aa-b09df076d5e6 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:21.761147 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707de0a-8b6e-45dd-bcec-59798f259327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:21.761781 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707de0a-8b6e-45dd-bcec-59798f259327 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:21.762055 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707de0a-8b6e-45dd-bcec-59798f259327 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:21.762328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707de0a-8b6e-45dd-bcec-59798f259327 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:21.763456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707de0a-8b6e-45dd-bcec-59798f259327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:21.764893 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707de0a-8b6e-45dd-bcec-59798f259327 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:21.766445 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 109/231] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:21.771720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707de0a-8b6e-45dd-bcec-59798f259327 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:21.780770 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:21.839806 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:21.843313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88e5c9-71aa-4e78-a8aa-b09df076d5e6 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:21.874241 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707de0a-8b6e-45dd-bcec-59798f259327 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:21.878852 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b8df9-0140-4788-be7c-5602a2a85dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:21.879480 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b8df9-0140-4788-be7c-5602a2a85dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:21.879759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b8df9-0140-4788-be7c-5602a2a85dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:21.880025 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b8df9-0140-4788-be7c-5602a2a85dc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:21.880677 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b8df9-0140-4788-be7c-5602a2a85dc9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:21.880973 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b8df9-0140-4788-be7c-5602a2a85dc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:21.882364 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b8df9-0140-4788-be7c-5602a2a85dc9 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:21.890901 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 123/232] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:26:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:21.894825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b8df9-0140-4788-be7c-5602a2a85dc9 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:21.907990 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff9779-ee52-48b0-9ed5-4dc640b5aea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:21.908598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff9779-ee52-48b0-9ed5-4dc640b5aea1 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:21.908875 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff9779-ee52-48b0-9ed5-4dc640b5aea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:21.909144 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff9779-ee52-48b0-9ed5-4dc640b5aea1 None admin] PATH_INFO: `/v3/groups` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:21.909779 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff9779-ee52-48b0-9ed5-4dc640b5aea1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:21.911238 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff9779-ee52-48b0-9ed5-4dc640b5aea1 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:21.926472 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 110/233] 10.222.0.16 () {60 vars in 1134 bytes} [Wed Oct 12 14:26:21 2022] POST /identity/v3/groups => generated 229 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 14:26:21.927306 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff9779-ee52-48b0-9ed5-4dc640b5aea1 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:22.002212 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b8df9-0140-4788-be7c-5602a2a85dc9 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:22.012020 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:22.041005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f36470-179d-45a8-803f-4ce96d76728a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:22.041642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f36470-179d-45a8-803f-4ce96d76728a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:22.041918 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f36470-179d-45a8-803f-4ce96d76728a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:22.042166 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f36470-179d-45a8-803f-4ce96d76728a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:22.053031 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff9779-ee52-48b0-9ed5-4dc640b5aea1 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:22.054358 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f36470-179d-45a8-803f-4ce96d76728a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:22.060547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-f0f36470-179d-45a8-803f-4ce96d76728a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:22.060547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:22.060547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:22.060547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:22.060547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:22.060547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:22.060547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:22.060547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:22.060547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:22.060547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:22.060547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:22.060547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:22.060547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:22.060547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:22.060547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:22.060547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:22.061488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:22.061488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:22.061488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:22.061488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:22.061488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:22.061488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:22.061488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:22.061488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:22.061488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:22.061488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:22.061488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:22.061488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:22.061488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:22.061488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:22.061488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:22.061488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:22.062243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:22.062243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:22.062243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:22.062243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:22.062243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:22.062243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:22.062243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:22.062243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:22.062243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:22.062243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:22.062243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:22.062243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:22.062243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 124/234] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:22.069028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f36470-179d-45a8-803f-4ce96d76728a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:22.096960 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcba58-2c58-43eb-a740-fe1ebd80920b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:22.097637 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcba58-2c58-43eb-a740-fe1ebd80920b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:22.098325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcba58-2c58-43eb-a740-fe1ebd80920b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:22.098435 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:22.098512 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcba58-2c58-43eb-a740-fe1ebd80920b None admin] PATH_INFO: `/v3/users/c34ada690c984d6290f49dbc81cbf35d` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:22.116531 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bcba58-2c58-43eb-a740-fe1ebd80920b None admin] RBAC: Authorizing `identity:get_user(user_id=c34ada690c984d6290f49dbc81cbf35d)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:22.118061 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bcba58-2c58-43eb-a740-fe1ebd80920b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:22.119801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 111/235] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:21 2022] GET /identity/v3/users/c34ada690c984d6290f49dbc81cbf35d => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:22.125522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcba58-2c58-43eb-a740-fe1ebd80920b None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:22.185911 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f36470-179d-45a8-803f-4ce96d76728a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:22.226435 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeaba6-607e-49d7-aa11-95098318a09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:22.227616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeaba6-607e-49d7-aa11-95098318a09e None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:22.228013 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeaba6-607e-49d7-aa11-95098318a09e None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:22.228282 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeaba6-607e-49d7-aa11-95098318a09e None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:22.228941 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efeaba6-607e-49d7-aa11-95098318a09e None admin] RBAC: Adding query filter params (name=member) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:22.229234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efeaba6-607e-49d7-aa11-95098318a09e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:22.230614 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efeaba6-607e-49d7-aa11-95098318a09e None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:22.238869 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 125/236] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:26:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:22.240093 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcba58-2c58-43eb-a740-fe1ebd80920b None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:22.244464 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeaba6-607e-49d7-aa11-95098318a09e None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:22.267454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:22.280380 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b7292-3a7c-4612-995e-7804ca721ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:22.280991 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b7292-3a7c-4612-995e-7804ca721ebd None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:22.281259 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b7292-3a7c-4612-995e-7804ca721ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:22.281523 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b7292-3a7c-4612-995e-7804ca721ebd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:22.282288 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b7292-3a7c-4612-995e-7804ca721ebd None admin] RBAC: Adding query filter params (role.id=ac952ec5ebb24b2fa6476a7990e40147, user.id=c34ada690c984d6290f49dbc81cbf35d, scope.system=all) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:22.282591 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b7292-3a7c-4612-995e-7804ca721ebd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:22.284159 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b7292-3a7c-4612-995e-7804ca721ebd None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:22.290149 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 112/237] 10.222.0.16 () {56 vars in 1308 bytes} [Wed Oct 12 14:26:22 2022] GET /identity/v3/role_assignments?user.id=c34ada690c984d6290f49dbc81cbf35d&scope.system=all&role.id=ac952ec5ebb24b2fa6476a7990e40147 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:22.334598 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:22.353632 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeaba6-607e-49d7-aa11-95098318a09e None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:22.391071 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544a5e3-0f67-4c11-8a62-252f2ee981c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:22.391761 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544a5e3-0f67-4c11-8a62-252f2ee981c5 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:22.392028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544a5e3-0f67-4c11-8a62-252f2ee981c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:22.392298 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544a5e3-0f67-4c11-8a62-252f2ee981c5 None admin] PATH_INFO: `/v3/users/1545576dc9de4dedb41dd5712d682a09` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:22.393468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544a5e3-0f67-4c11-8a62-252f2ee981c5 None admin] RBAC: Authorizing `identity:get_user(user_id=1545576dc9de4dedb41dd5712d682a09)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:22.394982 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544a5e3-0f67-4c11-8a62-252f2ee981c5 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:22.396840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 126/238] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:22 2022] GET /identity/v3/users/1545576dc9de4dedb41dd5712d682a09 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:22.403413 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b7292-3a7c-4612-995e-7804ca721ebd None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:22.513922 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b7292-3a7c-4612-995e-7804ca721ebd None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:22.549207 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86105e-4399-428f-ae8b-f84172cc395e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:22.549780 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86105e-4399-428f-ae8b-f84172cc395e None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:22.550055 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86105e-4399-428f-ae8b-f84172cc395e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:22.550326 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86105e-4399-428f-ae8b-f84172cc395e None admin] PATH_INFO: `/v3/system/users/1545576dc9de4dedb41dd5712d682a09/roles/6d1894444bfd400a89591929bd8925de` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:22.551751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea86105e-4399-428f-ae8b-f84172cc395e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1545576dc9de4dedb41dd5712d682a09, role_id=6d1894444bfd400a89591929bd8925de)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:22.553134 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea86105e-4399-428f-ae8b-f84172cc395e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:22.562071 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 113/239] 10.222.0.16 () {58 vars in 1334 bytes} [Wed Oct 12 14:26:22 2022] PUT /identity/v3/system/users/1545576dc9de4dedb41dd5712d682a09/roles/6d1894444bfd400a89591929bd8925de => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:26:24.136434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f03b67-4435-4d3a-bca4-08cfe450c3c5 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:24.136932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f03b67-4435-4d3a-bca4-08cfe450c3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:24.137005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f03b67-4435-4d3a-bca4-08cfe450c3c5 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:24.138478 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 127/240] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:24.143752 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c8c72-6c6c-4376-aae0-69389aafca6d None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:24.144195 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c8c72-6c6c-4376-aae0-69389aafca6d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:24.144337 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c8c72-6c6c-4376-aae0-69389aafca6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:24.166614 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-b29c8c72-6c6c-4376-aae0-69389aafca6d None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:24.167855 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-b29c8c72-6c6c-4376-aae0-69389aafca6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:24.216897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:24.239190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 114/241] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:24.449082 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e386b-dcd4-4c5b-997c-07533ad724dd None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:24.449431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e386b-dcd4-4c5b-997c-07533ad724dd None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:24.449632 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e386b-dcd4-4c5b-997c-07533ad724dd None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:24.451084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 128/242] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:24.456083 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690c90-fb65-464e-b6ef-e71d1be09466 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:24.456369 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690c90-fb65-464e-b6ef-e71d1be09466 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:24.456638 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690c90-fb65-464e-b6ef-e71d1be09466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:24.475345 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c9690c90-fb65-464e-b6ef-e71d1be09466 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:24.476521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c9690c90-fb65-464e-b6ef-e71d1be09466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:24.511483 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e386b-dcd4-4c5b-997c-07533ad724dd None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:24.546481 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 115/243] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:24.630391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e386b-dcd4-4c5b-997c-07533ad724dd None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:24.665704 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb58a90-a6be-4fd5-b0a2-0969ac854452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:24.666334 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb58a90-a6be-4fd5-b0a2-0969ac854452 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:24.666610 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb58a90-a6be-4fd5-b0a2-0969ac854452 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:24.666881 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb58a90-a6be-4fd5-b0a2-0969ac854452 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:24.668026 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb58a90-a6be-4fd5-b0a2-0969ac854452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:24.669476 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb58a90-a6be-4fd5-b0a2-0969ac854452 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:24.671222 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 129/244] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:24.682042 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9690c90-fb65-464e-b6ef-e71d1be09466 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:24.726194 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74330c2e-e88f-41f1-9d13-d64bfe9b10e3 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:24.726401 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74330c2e-e88f-41f1-9d13-d64bfe9b10e3 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:24.727222 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74330c2e-e88f-41f1-9d13-d64bfe9b10e3 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:24.728557 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 130/245] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:26:24.733898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb53eb-8571-4ce2-ac6f-5fa1fbf45898 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:24.734327 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb53eb-8571-4ce2-ac6f-5fa1fbf45898 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:24.735038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb53eb-8571-4ce2-ac6f-5fa1fbf45898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:24.754894 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-25bb53eb-8571-4ce2-ac6f-5fa1fbf45898 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:24.756088 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-25bb53eb-8571-4ce2-ac6f-5fa1fbf45898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:24.771779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:24.788856 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9690c90-fb65-464e-b6ef-e71d1be09466 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:24.802651 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:24.824919 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17c6c3-8a71-41d6-8fb5-fb9274f30685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:24.825776 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17c6c3-8a71-41d6-8fb5-fb9274f30685 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:24.826231 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17c6c3-8a71-41d6-8fb5-fb9274f30685 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:24.826702 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17c6c3-8a71-41d6-8fb5-fb9274f30685 None admin] PATH_INFO: `/v3/groups` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:24.827545 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17c6c3-8a71-41d6-8fb5-fb9274f30685 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:24.829169 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17c6c3-8a71-41d6-8fb5-fb9274f30685 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:24.831273 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 131/246] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:26:24.835551 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb53eb-8571-4ce2-ac6f-5fa1fbf45898 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:24.855182 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 116/247] 10.222.0.16 () {60 vars in 1134 bytes} [Wed Oct 12 14:26:24 2022] POST /identity/v3/groups => generated 226 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:26:24.948932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb53eb-8571-4ce2-ac6f-5fa1fbf45898 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:24.986733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcaded1-4289-495d-ab8e-d7264e1a2a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:24.987326 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcaded1-4289-495d-ab8e-d7264e1a2a01 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:24.987653 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcaded1-4289-495d-ab8e-d7264e1a2a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:24.987946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcaded1-4289-495d-ab8e-d7264e1a2a01 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:24.999911 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcaded1-4289-495d-ab8e-d7264e1a2a01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:25.005135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-8dcaded1-4289-495d-ab8e-d7264e1a2a01 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 14:26:25.005135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:25.005135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:25.005135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:25.005135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:25.005135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:25.005135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:25.005135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:25.005135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:25.005135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:25.005135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:25.005135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:25.005135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:25.005135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:25.005135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:25.005135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:25.005995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:25.005995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:25.005995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:25.005995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:25.005995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:25.005995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:25.005995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:25.005995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:25.005995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:25.005995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:25.005995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:25.005995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:25.005995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:25.005995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:25.005995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:25.005995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:25.006759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:25.006759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:25.006759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:25.006759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:25.006759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:25.006759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:25.006759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:25.006759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:25.006759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:25.006759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:25.006759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 14:26:25.006759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:25.013054 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 132/248] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:25.013200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17c6c3-8a71-41d6-8fb5-fb9274f30685 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:25.107959 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcaded1-4289-495d-ab8e-d7264e1a2a01 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:25.119047 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17c6c3-8a71-41d6-8fb5-fb9274f30685 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:25.123466 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:25.165188 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8bd9b-8262-46cc-a540-026318e8b4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:25.165990 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8bd9b-8262-46cc-a540-026318e8b4e1 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:25.166265 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8bd9b-8262-46cc-a540-026318e8b4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:25.166534 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8bd9b-8262-46cc-a540-026318e8b4e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:25.167495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae8bd9b-8262-46cc-a540-026318e8b4e1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:25.167623 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae8bd9b-8262-46cc-a540-026318e8b4e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:25.168902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae8bd9b-8262-46cc-a540-026318e8b4e1 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:25.178862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 117/249] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:26:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:25.189581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8bd9b-8262-46cc-a540-026318e8b4e1 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:25.231299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:25.247271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcaded1-4289-495d-ab8e-d7264e1a2a01 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:25.288273 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65458c7c-da6b-4c11-90fd-01d351825d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:25.290117 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65458c7c-da6b-4c11-90fd-01d351825d5a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:25.290677 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65458c7c-da6b-4c11-90fd-01d351825d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:25.291084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65458c7c-da6b-4c11-90fd-01d351825d5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:25.303674 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65458c7c-da6b-4c11-90fd-01d351825d5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:25.310063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-65458c7c-da6b-4c11-90fd-01d351825d5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:25.310063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:25.310063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:25.310063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:25.310063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:25.310063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:25.310063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:25.310063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:25.310063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:25.310063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:25.310063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:25.310063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:25.310063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:25.310063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:25.310063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:25.310063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:25.311169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:25.311169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:25.311169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:25.311169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:25.311169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:25.311169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:25.311169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:25.311169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:25.311169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:25.311169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:25.311169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:25.311169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:25.311169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:25.311169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:25.311169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:25.311169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:25.311946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:25.311946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:25.311946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:25.311946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:25.311946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:25.311946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:25.311946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:25.311946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:25.311946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:25.311946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:25.311946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:25.311946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:25.314393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 133/250] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:25.316708 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8bd9b-8262-46cc-a540-026318e8b4e1 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:25.321029 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65458c7c-da6b-4c11-90fd-01d351825d5a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:25.355403 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4276e-928a-475e-b6e1-dac4d6a1b551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:25.356146 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4276e-928a-475e-b6e1-dac4d6a1b551 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:25.356429 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4276e-928a-475e-b6e1-dac4d6a1b551 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:25.356705 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4276e-928a-475e-b6e1-dac4d6a1b551 None admin] PATH_INFO: `/v3/users/c34ada690c984d6290f49dbc81cbf35d` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:25.357861 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4276e-928a-475e-b6e1-dac4d6a1b551 None admin] RBAC: Authorizing `identity:get_user(user_id=c34ada690c984d6290f49dbc81cbf35d)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:25.359421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4276e-928a-475e-b6e1-dac4d6a1b551 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:25.361998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 118/251] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:25 2022] GET /identity/v3/users/c34ada690c984d6290f49dbc81cbf35d => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:25.368209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4276e-928a-475e-b6e1-dac4d6a1b551 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:25.397983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:25.433408 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65458c7c-da6b-4c11-90fd-01d351825d5a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:25.470854 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015e05d-7e77-4fdc-87f0-af8da6f4131a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:25.471804 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015e05d-7e77-4fdc-87f0-af8da6f4131a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:25.472245 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015e05d-7e77-4fdc-87f0-af8da6f4131a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:25.472677 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015e05d-7e77-4fdc-87f0-af8da6f4131a None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:25.473501 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015e05d-7e77-4fdc-87f0-af8da6f4131a None admin] RBAC: Adding query filter params (name=member) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:25.473963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015e05d-7e77-4fdc-87f0-af8da6f4131a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:25.475566 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015e05d-7e77-4fdc-87f0-af8da6f4131a None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:25.486205 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 134/252] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:26:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:25.490870 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4276e-928a-475e-b6e1-dac4d6a1b551 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:25.492002 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015e05d-7e77-4fdc-87f0-af8da6f4131a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:25.507725 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:25.531375 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8587a4f-b091-402d-9a44-effeea095c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:25.532127 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8587a4f-b091-402d-9a44-effeea095c35 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:25.532478 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8587a4f-b091-402d-9a44-effeea095c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:25.532829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8587a4f-b091-402d-9a44-effeea095c35 None admin] PATH_INFO: `/v3/system/users/c34ada690c984d6290f49dbc81cbf35d/roles/ac952ec5ebb24b2fa6476a7990e40147` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:25.534411 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8587a4f-b091-402d-9a44-effeea095c35 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c34ada690c984d6290f49dbc81cbf35d, role_id=ac952ec5ebb24b2fa6476a7990e40147)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:25.535979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8587a4f-b091-402d-9a44-effeea095c35 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:25.545478 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 119/253] 10.222.0.16 () {58 vars in 1334 bytes} [Wed Oct 12 14:26:25 2022] PUT /identity/v3/system/users/c34ada690c984d6290f49dbc81cbf35d/roles/ac952ec5ebb24b2fa6476a7990e40147 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:26:25.607540 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015e05d-7e77-4fdc-87f0-af8da6f4131a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:25.642370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:25.647854 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284496f-4ea7-4b6b-81b3-b805f7b2ef28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:25.650484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284496f-4ea7-4b6b-81b3-b805f7b2ef28 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:25.650766 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284496f-4ea7-4b6b-81b3-b805f7b2ef28 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:25.651039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284496f-4ea7-4b6b-81b3-b805f7b2ef28 None admin] PATH_INFO: `/v3/users/1545576dc9de4dedb41dd5712d682a09` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:25.652540 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284496f-4ea7-4b6b-81b3-b805f7b2ef28 None admin] RBAC: Authorizing `identity:get_user(user_id=1545576dc9de4dedb41dd5712d682a09)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:25.654524 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284496f-4ea7-4b6b-81b3-b805f7b2ef28 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:25.656332 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 135/254] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:25 2022] GET /identity/v3/users/1545576dc9de4dedb41dd5712d682a09 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:25.662585 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8587a4f-b091-402d-9a44-effeea095c35 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:25.792141 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8587a4f-b091-402d-9a44-effeea095c35 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:25.831356 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255ecea-361f-4652-97bf-8760e6db9f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:25.832022 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255ecea-361f-4652-97bf-8760e6db9f51 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:25.832307 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255ecea-361f-4652-97bf-8760e6db9f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:25.832578 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255ecea-361f-4652-97bf-8760e6db9f51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:25.833472 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255ecea-361f-4652-97bf-8760e6db9f51 None admin] RBAC: Adding query filter params (role.id=6d1894444bfd400a89591929bd8925de, user.id=1545576dc9de4dedb41dd5712d682a09, scope.system=all) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:25.834008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255ecea-361f-4652-97bf-8760e6db9f51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:25.836510 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255ecea-361f-4652-97bf-8760e6db9f51 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:25.850554 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 120/255] 10.222.0.16 () {56 vars in 1308 bytes} [Wed Oct 12 14:26:25 2022] GET /identity/v3/role_assignments?user.id=1545576dc9de4dedb41dd5712d682a09&scope.system=all&role.id=6d1894444bfd400a89591929bd8925de => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:27.565398 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6964e3-6823-44a8-8e2e-fa4e57195a34 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:27.565902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6964e3-6823-44a8-8e2e-fa4e57195a34 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:27.565975 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6964e3-6823-44a8-8e2e-fa4e57195a34 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:27.567566 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 136/256] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:27.579013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f07dbd-07bd-4ccc-a322-473c77fea618 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:27.579516 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f07dbd-07bd-4ccc-a322-473c77fea618 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:27.579588 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f07dbd-07bd-4ccc-a322-473c77fea618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:27.604980 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-94f07dbd-07bd-4ccc-a322-473c77fea618 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:27.606162 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-94f07dbd-07bd-4ccc-a322-473c77fea618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:27.636932 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:27.686913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 121/257] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 12 14:26:27.992698 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39818e1-7c30-48dc-98e2-4a75bef50055 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:27.992960 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39818e1-7c30-48dc-98e2-4a75bef50055 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:27.993310 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39818e1-7c30-48dc-98e2-4a75bef50055 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:27.994838 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 137/258] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:27.999973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d30b1-5a74-4e62-bac6-538ed519e643 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:28.000258 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d30b1-5a74-4e62-bac6-538ed519e643 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:28.000531 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d30b1-5a74-4e62-bac6-538ed519e643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:28.024674 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-1c0d30b1-5a74-4e62-bac6-538ed519e643 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:28.025880 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-1c0d30b1-5a74-4e62-bac6-538ed519e643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:28.036451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 122/259] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 14:26:28.040682 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39818e1-7c30-48dc-98e2-4a75bef50055 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:28.064379 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39818e1-7c30-48dc-98e2-4a75bef50055 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:28.068866 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf0a1b-ca97-459d-ae97-50f18f4b7aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:28.069497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf0a1b-ca97-459d-ae97-50f18f4b7aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:28.069835 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf0a1b-ca97-459d-ae97-50f18f4b7aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:28.070169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf0a1b-ca97-459d-ae97-50f18f4b7aa2 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:28.092072 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf0a1b-ca97-459d-ae97-50f18f4b7aa2 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:28.096997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-e3cf0a1b-ca97-459d-ae97-50f18f4b7aa2 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:28.096997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:28.096997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:28.096997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:28.096997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:28.096997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:28.096997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:28.096997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:28.096997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:28.096997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:28.096997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:28.096997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:28.096997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:28.096997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:28.096997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:28.096997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:28.097839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:28.097839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:28.097839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:28.097839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:28.097839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:28.097839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:28.097839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:28.097839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:28.097839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:28.097839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:28.097839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:28.097839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:28.097839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:28.097839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:28.097839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:28.097839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:28.098595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:28.098595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:28.098595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:28.098595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:28.098595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:28.098595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:28.098595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:28.098595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:28.098595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:28.098595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:28.098595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:28.098595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:28.098595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 138/260] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Oct 12 14:26:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 58 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:28.105990 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d30b1-5a74-4e62-bac6-538ed519e643 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:28.118633 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d30b1-5a74-4e62-bac6-538ed519e643 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:28.126788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad434ba-1851-4b8f-a696-9122b6068769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:28.127444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad434ba-1851-4b8f-a696-9122b6068769 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:28.127725 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad434ba-1851-4b8f-a696-9122b6068769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:28.127999 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad434ba-1851-4b8f-a696-9122b6068769 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:28.128681 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad434ba-1851-4b8f-a696-9122b6068769 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:28.128979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad434ba-1851-4b8f-a696-9122b6068769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:28.130763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: WARNING py.warnings [None req-fad434ba-1851-4b8f-a696-9122b6068769 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 12 14:26:28.130763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: warnings.warn(msg) Oct 12 14:26:28.130763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: Oct 12 14:26:28.131714 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad434ba-1851-4b8f-a696-9122b6068769 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:28.146806 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 123/261] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Oct 12 14:26:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:28.153525 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf0a1b-ca97-459d-ae97-50f18f4b7aa2 admin admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:28.169180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf0a1b-ca97-459d-ae97-50f18f4b7aa2 admin admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:28.179667 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3af6c9-a4f4-483d-b0a4-7bb3422a140a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:28.180454 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3af6c9-a4f4-483d-b0a4-7bb3422a140a admin admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:28.180769 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3af6c9-a4f4-483d-b0a4-7bb3422a140a admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:28.181041 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3af6c9-a4f4-483d-b0a4-7bb3422a140a admin admin] PATH_INFO: `/v3/groups/8e91e75a55854d5bb3ad03309346b75b` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:28.196194 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3af6c9-a4f4-483d-b0a4-7bb3422a140a admin admin] RBAC: Authorizing `identity:get_group(group_id=8e91e75a55854d5bb3ad03309346b75b)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:28.198035 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: WARNING py.warnings [None req-ff3af6c9-a4f4-483d-b0a4-7bb3422a140a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 14:26:28.198035 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: warnings.warn(msg) Oct 12 14:26:28.198035 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: Oct 12 14:26:28.199338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3af6c9-a4f4-483d-b0a4-7bb3422a140a admin admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:28.201037 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 139/262] 10.222.0.16 () {56 vars in 1201 bytes} [Wed Oct 12 14:26:28 2022] GET /identity/v3/groups/8e91e75a55854d5bb3ad03309346b75b => generated 226 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:28.206977 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad434ba-1851-4b8f-a696-9122b6068769 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:28.224553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad434ba-1851-4b8f-a696-9122b6068769 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:28.232297 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e34d0-071a-4931-bc89-d8974072a206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:28.232964 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e34d0-071a-4931-bc89-d8974072a206 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:28.233241 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e34d0-071a-4931-bc89-d8974072a206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:28.233518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e34d0-071a-4931-bc89-d8974072a206 admin admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:28.238320 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e34d0-071a-4931-bc89-d8974072a206 admin admin] RBAC: Authorizing `identity:get_project(project_id=ad7b7e0473f64a4685d43fc69df8b643)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:28.240465 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e34d0-071a-4931-bc89-d8974072a206 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:28.242395 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 124/263] 10.222.0.16 () {56 vars in 1207 bytes} [Wed Oct 12 14:26:28 2022] GET /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643 => generated 298 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:28.248949 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3af6c9-a4f4-483d-b0a4-7bb3422a140a admin admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:28.264506 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3af6c9-a4f4-483d-b0a4-7bb3422a140a admin admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:28.275849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9c54e-f10c-4d36-9c52-da7522d80e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:28.276525 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9c54e-f10c-4d36-9c52-da7522d80e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:28.276798 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9c54e-f10c-4d36-9c52-da7522d80e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:28.277066 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9c54e-f10c-4d36-9c52-da7522d80e45 admin admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643/groups/8e91e75a55854d5bb3ad03309346b75b/roles/6d1894444bfd400a89591929bd8925de` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:28.284415 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9c54e-f10c-4d36-9c52-da7522d80e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad7b7e0473f64a4685d43fc69df8b643, group_id=8e91e75a55854d5bb3ad03309346b75b, role_id=6d1894444bfd400a89591929bd8925de)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:28.285444 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: WARNING py.warnings [None req-9cf9c54e-f10c-4d36-9c52-da7522d80e45 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 14:26:28.285444 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: warnings.warn(msg) Oct 12 14:26:28.285444 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: Oct 12 14:26:28.287428 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9c54e-f10c-4d36-9c52-da7522d80e45 admin admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:28.312935 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 140/264] 10.222.0.16 () {58 vars in 1463 bytes} [Wed Oct 12 14:26:28 2022] PUT /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643/groups/8e91e75a55854d5bb3ad03309346b75b/roles/6d1894444bfd400a89591929bd8925de => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:26:28.400206 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e34d0-071a-4931-bc89-d8974072a206 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:28.410465 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e34d0-071a-4931-bc89-d8974072a206 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:28.484931 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e5a57-5d77-48b9-b80a-3db76f30835a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:28.485593 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e5a57-5d77-48b9-b80a-3db76f30835a admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:28.485897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e5a57-5d77-48b9-b80a-3db76f30835a admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:28.486142 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e5a57-5d77-48b9-b80a-3db76f30835a admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:28.505098 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e5a57-5d77-48b9-b80a-3db76f30835a admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:28.510953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-010e5a57-5d77-48b9-b80a-3db76f30835a admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:26:28.510953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:28.510953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:28.510953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:28.510953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:28.510953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:28.510953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:28.510953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:28.510953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:28.510953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:28.510953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:28.510953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:28.510953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:28.510953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:28.510953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:28.510953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:28.513176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:28.513176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:28.513176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:28.513176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:28.513176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:28.513176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:28.513176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:28.513176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:28.513176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:28.513176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:28.513176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:28.513176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:28.513176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:28.513176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:28.513176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:28.513176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:28.515387 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:28.515387 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:28.515387 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:28.515387 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:28.515387 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:28.515387 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:28.515387 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:28.515387 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:28.515387 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:28.515387 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:28.515387 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:26:28.515387 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:28.515387 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 125/265] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Oct 12 14:26:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 113 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:28.516995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31cc79-331b-436d-814c-7330624fd858 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:28.516995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31cc79-331b-436d-814c-7330624fd858 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:28.516995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31cc79-331b-436d-814c-7330624fd858 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:28.517614 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 141/266] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:28.519720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e5a57-5d77-48b9-b80a-3db76f30835a admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:28.524942 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f8054-af61-4d0f-bf9c-0e84613baae9 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:28.525201 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f8054-af61-4d0f-bf9c-0e84613baae9 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:28.525476 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f8054-af61-4d0f-bf9c-0e84613baae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:28.534876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e5a57-5d77-48b9-b80a-3db76f30835a admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:28.542029 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57365f5-2538-4617-a64a-fec49a2b5315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:28.542652 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57365f5-2538-4617-a64a-fec49a2b5315 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:28.542931 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57365f5-2538-4617-a64a-fec49a2b5315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:28.543226 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57365f5-2538-4617-a64a-fec49a2b5315 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:28.543904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57365f5-2538-4617-a64a-fec49a2b5315 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:28.544205 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57365f5-2538-4617-a64a-fec49a2b5315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:28.545842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57365f5-2538-4617-a64a-fec49a2b5315 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:28.551361 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-259f8054-af61-4d0f-bf9c-0e84613baae9 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:28.552997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-259f8054-af61-4d0f-bf9c-0e84613baae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:28.561725 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 126/267] 10.222.0.16 () {56 vars in 1132 bytes} [Wed Oct 12 14:26:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:28.566024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 142/268] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 14:26:28.571325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57365f5-2538-4617-a64a-fec49a2b5315 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:28.589590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57365f5-2538-4617-a64a-fec49a2b5315 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:28.603869 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12452785-8a61-42cf-b997-45b6f24e8e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:28.604518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12452785-8a61-42cf-b997-45b6f24e8e8d admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:28.604797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12452785-8a61-42cf-b997-45b6f24e8e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:28.605072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12452785-8a61-42cf-b997-45b6f24e8e8d admin admin] PATH_INFO: `/v3/groups/8e91e75a55854d5bb3ad03309346b75b` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:28.606244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12452785-8a61-42cf-b997-45b6f24e8e8d admin admin] RBAC: Authorizing `identity:get_group(group_id=8e91e75a55854d5bb3ad03309346b75b)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:28.607303 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: WARNING py.warnings [None req-12452785-8a61-42cf-b997-45b6f24e8e8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 14:26:28.607303 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: warnings.warn(msg) Oct 12 14:26:28.607303 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: Oct 12 14:26:28.608532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12452785-8a61-42cf-b997-45b6f24e8e8d admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:28.610126 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 127/269] 10.222.0.16 () {56 vars in 1201 bytes} [Wed Oct 12 14:26:28 2022] GET /identity/v3/groups/8e91e75a55854d5bb3ad03309346b75b => generated 226 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:28.615856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f8054-af61-4d0f-bf9c-0e84613baae9 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:28.633252 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f8054-af61-4d0f-bf9c-0e84613baae9 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:28.641122 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294bca9-6b4a-4c91-af53-1b0802faaaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:28.642117 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294bca9-6b4a-4c91-af53-1b0802faaaa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:28.642620 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294bca9-6b4a-4c91-af53-1b0802faaaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:28.643106 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294bca9-6b4a-4c91-af53-1b0802faaaa0 admin admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:28.648675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294bca9-6b4a-4c91-af53-1b0802faaaa0 admin admin] RBAC: Authorizing `identity:get_project(project_id=ad7b7e0473f64a4685d43fc69df8b643)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:28.651099 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294bca9-6b4a-4c91-af53-1b0802faaaa0 admin admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:28.653124 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 143/270] 10.222.0.16 () {56 vars in 1207 bytes} [Wed Oct 12 14:26:28 2022] GET /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643 => generated 298 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:28.658798 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12452785-8a61-42cf-b997-45b6f24e8e8d admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:28.671078 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12452785-8a61-42cf-b997-45b6f24e8e8d admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:28.683823 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efaf03-71f8-47ed-af4e-34dfb68473b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:28.684518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efaf03-71f8-47ed-af4e-34dfb68473b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:28.684806 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efaf03-71f8-47ed-af4e-34dfb68473b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:28.685073 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efaf03-71f8-47ed-af4e-34dfb68473b7 admin admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643/groups/8e91e75a55854d5bb3ad03309346b75b/roles/f0761dd38b794ee6bfab00ac01f240cd` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:28.693385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efaf03-71f8-47ed-af4e-34dfb68473b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad7b7e0473f64a4685d43fc69df8b643, group_id=8e91e75a55854d5bb3ad03309346b75b, role_id=f0761dd38b794ee6bfab00ac01f240cd)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:28.694410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: WARNING py.warnings [None req-81efaf03-71f8-47ed-af4e-34dfb68473b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 14:26:28.694410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: warnings.warn(msg) Oct 12 14:26:28.694410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: Oct 12 14:26:28.696811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efaf03-71f8-47ed-af4e-34dfb68473b7 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:28.721880 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 128/271] 10.222.0.16 () {58 vars in 1463 bytes} [Wed Oct 12 14:26:28 2022] PUT /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643/groups/8e91e75a55854d5bb3ad03309346b75b/roles/f0761dd38b794ee6bfab00ac01f240cd => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:26:28.814201 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc466552-4662-478d-896d-20fc84cd770d None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:28.814374 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc466552-4662-478d-896d-20fc84cd770d None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:28.814670 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc466552-4662-478d-896d-20fc84cd770d None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:28.816352 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 144/272] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:28.822397 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b4d5e-5606-4de1-8efb-18e14554af71 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:28.822685 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b4d5e-5606-4de1-8efb-18e14554af71 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:28.823007 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b4d5e-5606-4de1-8efb-18e14554af71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:28.845094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d29b4d5e-5606-4de1-8efb-18e14554af71 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:28.846499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d29b4d5e-5606-4de1-8efb-18e14554af71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:28.921225 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:28.933969 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 129/273] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:28.957533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc466552-4662-478d-896d-20fc84cd770d None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:29.033076 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc466552-4662-478d-896d-20fc84cd770d None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:29.040136 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e06e1-ebab-4005-87a8-ff5b7c28bf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:29.041036 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e06e1-ebab-4005-87a8-ff5b7c28bf29 admin admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:29.041505 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e06e1-ebab-4005-87a8-ff5b7c28bf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:29.041983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e06e1-ebab-4005-87a8-ff5b7c28bf29 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:29.054645 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e06e1-ebab-4005-87a8-ff5b7c28bf29 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:29.059895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-fb9e06e1-ebab-4005-87a8-ff5b7c28bf29 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:29.059895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:29.059895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:29.059895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:29.059895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:29.059895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:29.059895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:29.059895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:29.059895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:29.059895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:29.059895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:29.059895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:29.059895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:29.059895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:29.059895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:29.059895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:29.060760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:29.060760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:29.060760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:29.060760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:29.060760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:29.060760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:29.060760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:29.060760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:29.060760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:29.060760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:29.060760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:29.060760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:29.060760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:29.060760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:29.060760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:29.060760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:29.061523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:29.061523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:29.061523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:29.061523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:29.061523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:29.061523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:29.061523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:29.061523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:29.061523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:29.061523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:29.061523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:29.061523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:29.064130 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 145/274] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Oct 12 14:26:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 107 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:29.071068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b4d5e-5606-4de1-8efb-18e14554af71 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:29.088072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b4d5e-5606-4de1-8efb-18e14554af71 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:29.097166 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac2d07-b882-4069-aa80-67b726531b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:29.097790 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac2d07-b882-4069-aa80-67b726531b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:29.098059 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac2d07-b882-4069-aa80-67b726531b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:29.098324 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac2d07-b882-4069-aa80-67b726531b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:29.098993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac2d07-b882-4069-aa80-67b726531b84 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:29.099339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac2d07-b882-4069-aa80-67b726531b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:29.101583 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac2d07-b882-4069-aa80-67b726531b84 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:29.115343 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 130/275] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Oct 12 14:26:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:29.121240 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e06e1-ebab-4005-87a8-ff5b7c28bf29 admin admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:29.136924 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e06e1-ebab-4005-87a8-ff5b7c28bf29 admin admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:29.144710 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767028a6-3f62-4eaf-bd3a-e08e4f3a0a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:29.145601 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767028a6-3f62-4eaf-bd3a-e08e4f3a0a47 admin admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:29.146094 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767028a6-3f62-4eaf-bd3a-e08e4f3a0a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:29.146622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767028a6-3f62-4eaf-bd3a-e08e4f3a0a47 admin admin] PATH_INFO: `/v3/groups/8e91e75a55854d5bb3ad03309346b75b` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:29.148030 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767028a6-3f62-4eaf-bd3a-e08e4f3a0a47 admin admin] RBAC: Authorizing `identity:get_group(group_id=8e91e75a55854d5bb3ad03309346b75b)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:29.150403 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767028a6-3f62-4eaf-bd3a-e08e4f3a0a47 admin admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:29.152533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 146/276] 10.222.0.16 () {56 vars in 1201 bytes} [Wed Oct 12 14:26:29 2022] GET /identity/v3/groups/8e91e75a55854d5bb3ad03309346b75b => generated 226 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:29.158188 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac2d07-b882-4069-aa80-67b726531b84 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:29.169156 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac2d07-b882-4069-aa80-67b726531b84 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:29.176805 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5fc9f-6998-4608-b032-c41e48a4f0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:29.177680 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5fc9f-6998-4608-b032-c41e48a4f0ba admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:29.178178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5fc9f-6998-4608-b032-c41e48a4f0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:29.178645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5fc9f-6998-4608-b032-c41e48a4f0ba admin admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:29.183891 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5fc9f-6998-4608-b032-c41e48a4f0ba admin admin] RBAC: Authorizing `identity:get_project(project_id=a78b76de457545f7927d98a7a89f4cec)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:29.186270 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5fc9f-6998-4608-b032-c41e48a4f0ba admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:29.188188 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 131/277] 10.222.0.16 () {56 vars in 1207 bytes} [Wed Oct 12 14:26:29 2022] GET /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec => generated 302 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:29.194016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767028a6-3f62-4eaf-bd3a-e08e4f3a0a47 admin admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:29.206360 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767028a6-3f62-4eaf-bd3a-e08e4f3a0a47 admin admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:29.215038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05622b8f-c587-4a57-be66-e50277b636c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:29.216026 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05622b8f-c587-4a57-be66-e50277b636c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:29.216496 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05622b8f-c587-4a57-be66-e50277b636c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:29.216967 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05622b8f-c587-4a57-be66-e50277b636c8 admin admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec/groups/8e91e75a55854d5bb3ad03309346b75b/roles/6d1894444bfd400a89591929bd8925de` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:29.219111 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05622b8f-c587-4a57-be66-e50277b636c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a78b76de457545f7927d98a7a89f4cec, group_id=8e91e75a55854d5bb3ad03309346b75b, role_id=6d1894444bfd400a89591929bd8925de)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:29.221999 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05622b8f-c587-4a57-be66-e50277b636c8 admin admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:29.247281 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 147/278] 10.222.0.16 () {58 vars in 1463 bytes} [Wed Oct 12 14:26:29 2022] PUT /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec/groups/8e91e75a55854d5bb3ad03309346b75b/roles/6d1894444bfd400a89591929bd8925de => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:26:29.302754 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5fc9f-6998-4608-b032-c41e48a4f0ba admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:29.396785 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fcc04-31ff-4e5e-9e25-e4bc8f88d518 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:29.397381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fcc04-31ff-4e5e-9e25-e4bc8f88d518 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:29.398396 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fcc04-31ff-4e5e-9e25-e4bc8f88d518 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:29.400781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 148/279] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:29.406146 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef605223-8ea6-43ea-b241-8f8c350235cf None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:29.406760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef605223-8ea6-43ea-b241-8f8c350235cf None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:29.409170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef605223-8ea6-43ea-b241-8f8c350235cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:29.443387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-ef605223-8ea6-43ea-b241-8f8c350235cf None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:29.445717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-ef605223-8ea6-43ea-b241-8f8c350235cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:29.480052 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5fc9f-6998-4608-b032-c41e48a4f0ba admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:29.529953 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 149/280] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 12 14:26:29.544002 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fdaac-1881-4298-998a-6d3eb9f6e688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:29.544694 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fdaac-1881-4298-998a-6d3eb9f6e688 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:29.544975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fdaac-1881-4298-998a-6d3eb9f6e688 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:29.545248 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fdaac-1881-4298-998a-6d3eb9f6e688 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:29.556729 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fdaac-1881-4298-998a-6d3eb9f6e688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:29.566595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-302fdaac-1881-4298-998a-6d3eb9f6e688 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 14:26:29.566595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:29.566595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:29.566595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:29.566595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:29.566595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:29.566595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:29.566595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:29.566595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:29.566595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:29.566595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:29.566595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:29.566595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:29.566595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:29.566595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:29.566595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:29.567522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:29.567522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:29.567522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:29.567522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:29.567522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:29.567522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:29.567522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:29.567522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:29.567522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:29.567522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:29.567522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:29.567522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:29.567522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:29.567522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:29.567522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:29.567522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:29.568299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:29.568299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:29.568299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:29.568299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:29.568299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:29.568299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:29.568299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:29.568299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:29.568299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:29.568299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:29.568299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 14:26:29.568299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:29.568299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 132/281] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:29.578806 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef605223-8ea6-43ea-b241-8f8c350235cf None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:29.633185 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:29.737385 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef605223-8ea6-43ea-b241-8f8c350235cf None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:29.783321 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d714ad-19df-4625-bd9b-8de4eb8adde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:29.783928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d714ad-19df-4625-bd9b-8de4eb8adde2 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:29.784199 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d714ad-19df-4625-bd9b-8de4eb8adde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:29.784467 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d714ad-19df-4625-bd9b-8de4eb8adde2 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:29.785125 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d714ad-19df-4625-bd9b-8de4eb8adde2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:29.785420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d714ad-19df-4625-bd9b-8de4eb8adde2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:29.786778 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d714ad-19df-4625-bd9b-8de4eb8adde2 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:29.795864 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 150/282] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:26:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:29.801671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fdaac-1881-4298-998a-6d3eb9f6e688 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:29.871619 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:29.951676 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fdaac-1881-4298-998a-6d3eb9f6e688 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:29.995082 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8746f0d-9cf3-45f7-a665-9e66bec59d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:29.996050 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8746f0d-9cf3-45f7-a665-9e66bec59d5b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:29.996500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8746f0d-9cf3-45f7-a665-9e66bec59d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:29.996942 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8746f0d-9cf3-45f7-a665-9e66bec59d5b None admin] PATH_INFO: `/v3/users/c34ada690c984d6290f49dbc81cbf35d` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:29.998272 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8746f0d-9cf3-45f7-a665-9e66bec59d5b None admin] RBAC: Authorizing `identity:get_user(user_id=c34ada690c984d6290f49dbc81cbf35d)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:30.000038 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8746f0d-9cf3-45f7-a665-9e66bec59d5b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:30.001850 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 133/283] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:26:29 2022] GET /identity/v3/users/c34ada690c984d6290f49dbc81cbf35d => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:30.008366 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8746f0d-9cf3-45f7-a665-9e66bec59d5b None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:30.008603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d714ad-19df-4625-bd9b-8de4eb8adde2 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:30.025836 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8746f0d-9cf3-45f7-a665-9e66bec59d5b None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:30.033597 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca655cd-978e-4ada-8c7f-3795c2af84eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:30.034236 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca655cd-978e-4ada-8c7f-3795c2af84eb admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:30.034513 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca655cd-978e-4ada-8c7f-3795c2af84eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:30.034782 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca655cd-978e-4ada-8c7f-3795c2af84eb admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:30.052933 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca655cd-978e-4ada-8c7f-3795c2af84eb admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:30.058642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-6ca655cd-978e-4ada-8c7f-3795c2af84eb admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:26:30.058642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:30.058642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:30.058642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:30.058642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:30.058642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:30.058642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:30.058642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:30.058642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:30.058642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:30.058642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:30.058642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:30.058642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:30.058642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:30.058642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:30.058642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:30.059584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:30.059584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:30.059584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:30.059584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:30.059584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:30.059584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:30.059584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:30.059584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:30.059584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:30.059584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:30.059584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:30.059584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:30.059584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:30.059584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:30.059584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:30.059584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:30.060376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:30.060376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:30.060376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:30.060376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:30.060376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:30.060376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:30.060376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:30.060376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:30.060376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:30.060376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:30.060376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:26:30.060376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:30.060376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 134/284] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Oct 12 14:26:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 52 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:30.065752 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca655cd-978e-4ada-8c7f-3795c2af84eb admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:30.078597 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca655cd-978e-4ada-8c7f-3795c2af84eb admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:30.089863 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb79b4-f135-49d5-83ec-fd6d64939221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:30.091368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb79b4-f135-49d5-83ec-fd6d64939221 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:30.092701 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb79b4-f135-49d5-83ec-fd6d64939221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:30.092980 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb79b4-f135-49d5-83ec-fd6d64939221 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:30.093659 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb79b4-f135-49d5-83ec-fd6d64939221 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:30.093960 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb79b4-f135-49d5-83ec-fd6d64939221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:30.097861 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb79b4-f135-49d5-83ec-fd6d64939221 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:30.113479 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 135/285] 10.222.0.16 () {56 vars in 1132 bytes} [Wed Oct 12 14:26:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:30.135254 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb79b4-f135-49d5-83ec-fd6d64939221 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:30.145772 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb79b4-f135-49d5-83ec-fd6d64939221 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:30.153746 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5a186-840c-4759-819a-90dcc2b1ce04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:30.154564 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5a186-840c-4759-819a-90dcc2b1ce04 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:30.154837 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5a186-840c-4759-819a-90dcc2b1ce04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:30.156235 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5a186-840c-4759-819a-90dcc2b1ce04 admin admin] PATH_INFO: `/v3/groups/8e91e75a55854d5bb3ad03309346b75b` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:30.159700 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc5a186-840c-4759-819a-90dcc2b1ce04 admin admin] RBAC: Authorizing `identity:get_group(group_id=8e91e75a55854d5bb3ad03309346b75b)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:30.161892 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc5a186-840c-4759-819a-90dcc2b1ce04 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:30.164504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 136/286] 10.222.0.16 () {56 vars in 1201 bytes} [Wed Oct 12 14:26:30 2022] GET /identity/v3/groups/8e91e75a55854d5bb3ad03309346b75b => generated 226 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:30.175160 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5a186-840c-4759-819a-90dcc2b1ce04 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:30.187750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5a186-840c-4759-819a-90dcc2b1ce04 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:30.194756 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6d6e1-403f-46e5-afa4-a6e89efbc91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:30.196990 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6d6e1-403f-46e5-afa4-a6e89efbc91c admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:30.197269 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6d6e1-403f-46e5-afa4-a6e89efbc91c admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:30.197543 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6d6e1-403f-46e5-afa4-a6e89efbc91c admin admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:30.200352 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:30.201380 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6d6e1-403f-46e5-afa4-a6e89efbc91c admin admin] RBAC: Authorizing `identity:get_project(project_id=a78b76de457545f7927d98a7a89f4cec)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:30.203878 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6d6e1-403f-46e5-afa4-a6e89efbc91c admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:30.205638 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 137/287] 10.222.0.16 () {56 vars in 1207 bytes} [Wed Oct 12 14:26:30 2022] GET /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec => generated 302 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:30.212407 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6d6e1-403f-46e5-afa4-a6e89efbc91c admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:30.218958 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d714ad-19df-4625-bd9b-8de4eb8adde2 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:30.222463 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6d6e1-403f-46e5-afa4-a6e89efbc91c admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:30.230245 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacf6ee-e8a4-447a-9618-4550332da9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:30.231229 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacf6ee-e8a4-447a-9618-4550332da9b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:30.231654 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacf6ee-e8a4-447a-9618-4550332da9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:30.232115 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacf6ee-e8a4-447a-9618-4550332da9b1 admin admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec/groups/8e91e75a55854d5bb3ad03309346b75b/roles/f0761dd38b794ee6bfab00ac01f240cd` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:30.234551 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacf6ee-e8a4-447a-9618-4550332da9b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a78b76de457545f7927d98a7a89f4cec, group_id=8e91e75a55854d5bb3ad03309346b75b, role_id=f0761dd38b794ee6bfab00ac01f240cd)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:30.237556 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacf6ee-e8a4-447a-9618-4550332da9b1 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:30.261251 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 138/288] 10.222.0.16 () {58 vars in 1463 bytes} [Wed Oct 12 14:26:30 2022] PUT /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec/groups/8e91e75a55854d5bb3ad03309346b75b/roles/f0761dd38b794ee6bfab00ac01f240cd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 14:26:30.263090 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26425c-e720-4ce3-8eeb-2df90579d939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:30.263735 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26425c-e720-4ce3-8eeb-2df90579d939 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:30.264007 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26425c-e720-4ce3-8eeb-2df90579d939 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:30.264323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26425c-e720-4ce3-8eeb-2df90579d939 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:30.265053 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26425c-e720-4ce3-8eeb-2df90579d939 None admin] RBAC: Adding query filter params (role.id=ac952ec5ebb24b2fa6476a7990e40147, user.id=c34ada690c984d6290f49dbc81cbf35d, scope.system=all) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:30.265359 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26425c-e720-4ce3-8eeb-2df90579d939 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:30.266831 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26425c-e720-4ce3-8eeb-2df90579d939 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:30.284459 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 151/289] 10.222.0.16 () {56 vars in 1308 bytes} [Wed Oct 12 14:26:30 2022] GET /identity/v3/role_assignments?user.id=c34ada690c984d6290f49dbc81cbf35d&scope.system=all&role.id=ac952ec5ebb24b2fa6476a7990e40147 => generated 510 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:31.232733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1409d8-c70a-422b-9585-911780aa313e None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:31.233604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1409d8-c70a-422b-9585-911780aa313e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:31.234019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1409d8-c70a-422b-9585-911780aa313e None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:31.235557 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 139/290] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:31.242491 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a9f00-f05f-4f0e-8ba4-37ffb7cdc7c4 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:31.242970 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a9f00-f05f-4f0e-8ba4-37ffb7cdc7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:31.243045 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a9f00-f05f-4f0e-8ba4-37ffb7cdc7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:31.268224 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-3d7a9f00-f05f-4f0e-8ba4-37ffb7cdc7c4 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:31.269491 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-3d7a9f00-f05f-4f0e-8ba4-37ffb7cdc7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:31.346929 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 152/291] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 12 14:26:31.888201 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1409d8-c70a-422b-9585-911780aa313e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:31.901514 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1409d8-c70a-422b-9585-911780aa313e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:31.903824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c77177-9a33-4ee8-bd75-9f2f34411fd9 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:31.904318 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c77177-9a33-4ee8-bd75-9f2f34411fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:31.904804 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c77177-9a33-4ee8-bd75-9f2f34411fd9 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:31.906340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 153/292] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:26:31.911556 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e92174-b974-43f2-9d70-d6584bc80e89 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:31.912056 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e92174-b974-43f2-9d70-d6584bc80e89 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:31.912520 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e92174-b974-43f2-9d70-d6584bc80e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:31.921911 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e17ce-44d0-4c38-b12b-d1fa7e000f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:31.923755 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e17ce-44d0-4c38-b12b-d1fa7e000f61 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:31.924252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e17ce-44d0-4c38-b12b-d1fa7e000f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:31.927492 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e17ce-44d0-4c38-b12b-d1fa7e000f61 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:31.942908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-51e92174-b974-43f2-9d70-d6584bc80e89 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:31.945178 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-51e92174-b974-43f2-9d70-d6584bc80e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:31.952286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e17ce-44d0-4c38-b12b-d1fa7e000f61 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:31.959263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-b17e17ce-44d0-4c38-b12b-d1fa7e000f61 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:26:31.959263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:31.959263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:31.959263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:31.959263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:31.959263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:31.959263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:31.959263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:31.959263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:31.959263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:31.959263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:31.959263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:31.959263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:31.959263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:31.959263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:31.959263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:31.960227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:31.960227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:31.960227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:31.960227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:31.960227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:31.960227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:31.960227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:31.960227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:31.960227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:31.960227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:31.960227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:31.960227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:31.960227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:31.960227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:31.960227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:31.960227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:31.960984 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:31.960984 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:31.960984 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:31.960984 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:31.960984 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:31.960984 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:31.960984 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:31.960984 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:31.960984 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:31.960984 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:31.960984 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:26:31.960984 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:31.963226 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 140/293] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Oct 12 14:26:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 76 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:31.964704 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19107d-2c45-471d-9d2d-c8f184ef58af None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:31.965182 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19107d-2c45-471d-9d2d-c8f184ef58af None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:31.965650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19107d-2c45-471d-9d2d-c8f184ef58af None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:31.971761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 141/294] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:31.973754 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19107d-2c45-471d-9d2d-c8f184ef58af None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:31.988384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19107d-2c45-471d-9d2d-c8f184ef58af None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:31.996247 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc7f44-ae72-4721-91b7-de574d00cd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:31.997113 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc7f44-ae72-4721-91b7-de574d00cd62 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:31.997601 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc7f44-ae72-4721-91b7-de574d00cd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:31.997938 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc7f44-ae72-4721-91b7-de574d00cd62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:31.998675 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc7f44-ae72-4721-91b7-de574d00cd62 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:31.999044 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc7f44-ae72-4721-91b7-de574d00cd62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:32.000766 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc7f44-ae72-4721-91b7-de574d00cd62 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:32.010272 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 142/295] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Oct 12 14:26:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:32.017114 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15fdbb-2075-47c4-9301-a27056d221ae None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:32.017401 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15fdbb-2075-47c4-9301-a27056d221ae None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:32.017674 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15fdbb-2075-47c4-9301-a27056d221ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:32.036599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 154/296] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:26:32.037348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3b15fdbb-2075-47c4-9301-a27056d221ae None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:32.038514 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e92174-b974-43f2-9d70-d6584bc80e89 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:32.039192 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3b15fdbb-2075-47c4-9301-a27056d221ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:32.050207 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e92174-b974-43f2-9d70-d6584bc80e89 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:32.055331 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81bc56c-8d06-4809-b019-0d2434b58974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:32.056238 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bc56c-8d06-4809-b019-0d2434b58974 admin admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:32.056709 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bc56c-8d06-4809-b019-0d2434b58974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:32.057321 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bc56c-8d06-4809-b019-0d2434b58974 admin admin] PATH_INFO: `/v3/groups/425459424c20415e9fc0bad2efd0958c` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:32.068539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81bc56c-8d06-4809-b019-0d2434b58974 admin admin] RBAC: Authorizing `identity:get_group(group_id=425459424c20415e9fc0bad2efd0958c)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:32.071448 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81bc56c-8d06-4809-b019-0d2434b58974 admin admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:32.074557 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 155/297] 10.222.0.16 () {56 vars in 1201 bytes} [Wed Oct 12 14:26:32 2022] GET /identity/v3/groups/425459424c20415e9fc0bad2efd0958c => generated 229 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:32.081768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81bc56c-8d06-4809-b019-0d2434b58974 admin admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:32.092364 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81bc56c-8d06-4809-b019-0d2434b58974 admin admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:32.096123 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:32.098079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5187a4-548e-4c10-95d3-5a20177f3a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:32.099019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5187a4-548e-4c10-95d3-5a20177f3a5b admin admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:32.099492 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5187a4-548e-4c10-95d3-5a20177f3a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:32.099903 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5187a4-548e-4c10-95d3-5a20177f3a5b admin admin] PATH_INFO: `/v3/projects/671bb1b187584e1b95d55373780c0171` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:32.103628 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5187a4-548e-4c10-95d3-5a20177f3a5b admin admin] RBAC: Authorizing `identity:get_project(project_id=671bb1b187584e1b95d55373780c0171)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:32.105642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5187a4-548e-4c10-95d3-5a20177f3a5b admin admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:32.110529 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 156/298] 10.222.0.16 () {56 vars in 1207 bytes} [Wed Oct 12 14:26:32 2022] GET /identity/v3/projects/671bb1b187584e1b95d55373780c0171 => generated 344 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:32.115415 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5187a4-548e-4c10-95d3-5a20177f3a5b admin admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:32.125677 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5187a4-548e-4c10-95d3-5a20177f3a5b admin admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:32.128363 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 143/299] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:32.134126 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73819efb-4b23-4365-9ffb-ae957001c459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:32.134999 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73819efb-4b23-4365-9ffb-ae957001c459 admin admin] REQUEST_METHOD: `PUT` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:32.135447 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73819efb-4b23-4365-9ffb-ae957001c459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:32.135879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73819efb-4b23-4365-9ffb-ae957001c459 admin admin] PATH_INFO: `/v3/projects/671bb1b187584e1b95d55373780c0171/groups/425459424c20415e9fc0bad2efd0958c/roles/4b1f48ab911e4fadbc70164ae43d508b` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:32.137931 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73819efb-4b23-4365-9ffb-ae957001c459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=671bb1b187584e1b95d55373780c0171, group_id=425459424c20415e9fc0bad2efd0958c, role_id=4b1f48ab911e4fadbc70164ae43d508b)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:32.140787 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73819efb-4b23-4365-9ffb-ae957001c459 admin admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:32.152891 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:32.163500 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 157/300] 10.222.0.16 () {58 vars in 1463 bytes} [Wed Oct 12 14:26:32 2022] PUT /identity/v3/projects/671bb1b187584e1b95d55373780c0171/groups/425459424c20415e9fc0bad2efd0958c/roles/4b1f48ab911e4fadbc70164ae43d508b => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 14:26:32.357545 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d4909-4845-4382-ad08-48c053ef54bd None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:32.358566 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d4909-4845-4382-ad08-48c053ef54bd None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:32.359045 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d4909-4845-4382-ad08-48c053ef54bd None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:32.360694 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 144/301] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:32.366986 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbfbed-c970-4488-b4e5-7d730aefa239 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:32.367583 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbfbed-c970-4488-b4e5-7d730aefa239 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:32.367583 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbfbed-c970-4488-b4e5-7d730aefa239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:32.390928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-80fbfbed-c970-4488-b4e5-7d730aefa239 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:32.392070 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-80fbfbed-c970-4488-b4e5-7d730aefa239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:32.440287 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d4909-4845-4382-ad08-48c053ef54bd None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:32.470815 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 158/302] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:32.553299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d4909-4845-4382-ad08-48c053ef54bd None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:32.589347 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbfbed-c970-4488-b4e5-7d730aefa239 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:32.589927 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646a716-c169-4dd7-aafb-244302d2ab61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:32.590762 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646a716-c169-4dd7-aafb-244302d2ab61 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:32.591261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646a716-c169-4dd7-aafb-244302d2ab61 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:32.591676 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646a716-c169-4dd7-aafb-244302d2ab61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:32.604503 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646a716-c169-4dd7-aafb-244302d2ab61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:32.609896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-0646a716-c169-4dd7-aafb-244302d2ab61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:32.609896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:32.609896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:32.609896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:32.609896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:32.609896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:32.609896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:32.609896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:32.609896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:32.609896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:32.609896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:32.609896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:32.609896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:32.609896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:32.609896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:32.609896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:32.610939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:32.610939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:32.610939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:32.610939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:32.610939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:32.610939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:32.610939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:32.610939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:32.610939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:32.610939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:32.610939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:32.610939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:32.610939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:32.610939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:32.610939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:32.610939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:32.611849 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:32.611849 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:32.611849 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:32.611849 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:32.611849 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:32.611849 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:32.611849 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:32.611849 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:32.611849 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:32.611849 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:32.611849 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:32.611849 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:32.613782 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 145/303] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:32.620286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646a716-c169-4dd7-aafb-244302d2ab61 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:32.659538 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:32.743107 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646a716-c169-4dd7-aafb-244302d2ab61 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:32.789924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec98c8-cc4c-47db-a5a5-2eae2a3304bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:32.790925 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec98c8-cc4c-47db-a5a5-2eae2a3304bb None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:32.791444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec98c8-cc4c-47db-a5a5-2eae2a3304bb None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:32.791965 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec98c8-cc4c-47db-a5a5-2eae2a3304bb None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:32.792992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec98c8-cc4c-47db-a5a5-2eae2a3304bb None admin] RBAC: Adding query filter params (name=member) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:32.793591 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec98c8-cc4c-47db-a5a5-2eae2a3304bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:32.799548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec98c8-cc4c-47db-a5a5-2eae2a3304bb None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:32.801896 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbfbed-c970-4488-b4e5-7d730aefa239 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:32.813728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 146/304] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:26:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:32.820685 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec98c8-cc4c-47db-a5a5-2eae2a3304bb None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:32.838737 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:32.841968 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd8c3c-fa3b-4f66-8343-945184385abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:32.842613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd8c3c-fa3b-4f66-8343-945184385abc None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:32.843905 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd8c3c-fa3b-4f66-8343-945184385abc None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:32.844182 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd8c3c-fa3b-4f66-8343-945184385abc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:32.855529 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd8c3c-fa3b-4f66-8343-945184385abc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:32.860510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-34dd8c3c-fa3b-4f66-8343-945184385abc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:26:32.860510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:32.860510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:32.860510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:32.860510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:32.860510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:32.860510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:32.860510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:32.860510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:32.860510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:32.860510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:32.860510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:32.860510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:32.860510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:32.860510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:32.860510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:32.861387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:32.861387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:32.861387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:32.861387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:32.861387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:32.861387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:32.861387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:32.861387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:32.861387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:32.861387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:32.861387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:32.861387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:32.861387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:32.861387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:32.861387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:32.861387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:32.862644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:32.862644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:32.862644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:32.862644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:32.862644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:32.862644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:32.862644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:32.862644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:32.862644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:32.862644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:32.862644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:26:32.862644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:32.862644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 159/305] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Oct 12 14:26:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:32.867740 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd8c3c-fa3b-4f66-8343-945184385abc None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:32.973255 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd8c3c-fa3b-4f66-8343-945184385abc None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:32.993666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec98c8-cc4c-47db-a5a5-2eae2a3304bb None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:33.005692 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:33.011656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d928056-a032-42a1-b286-ebaea5f2a38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:33.012271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d928056-a032-42a1-b286-ebaea5f2a38a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:33.012545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d928056-a032-42a1-b286-ebaea5f2a38a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:33.012814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d928056-a032-42a1-b286-ebaea5f2a38a None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:33.013482 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d928056-a032-42a1-b286-ebaea5f2a38a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:33.013782 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d928056-a032-42a1-b286-ebaea5f2a38a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:33.015176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d928056-a032-42a1-b286-ebaea5f2a38a None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:33.025589 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 160/306] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:33.026445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d928056-a032-42a1-b286-ebaea5f2a38a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:33.037348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed952e-2108-41ea-af5c-dd3e5edbe0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:33.038460 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed952e-2108-41ea-af5c-dd3e5edbe0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:33.039063 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed952e-2108-41ea-af5c-dd3e5edbe0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:33.039653 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed952e-2108-41ea-af5c-dd3e5edbe0b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:33.089609 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:33.136575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed952e-2108-41ea-af5c-dd3e5edbe0b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:33.138947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed952e-2108-41ea-af5c-dd3e5edbe0b6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:33.151343 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d928056-a032-42a1-b286-ebaea5f2a38a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:33.196119 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03f3de-761a-4e86-9e61-2274863f4239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:33.197061 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03f3de-761a-4e86-9e61-2274863f4239 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:33.197061 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03f3de-761a-4e86-9e61-2274863f4239 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:33.197304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03f3de-761a-4e86-9e61-2274863f4239 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:33.211368 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03f3de-761a-4e86-9e61-2274863f4239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:33.216827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-3b03f3de-761a-4e86-9e61-2274863f4239 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:33.216827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:33.216827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:33.216827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:33.216827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:33.216827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:33.216827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:33.216827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:33.216827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:33.216827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:33.216827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:33.216827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:33.216827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:33.216827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:33.216827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:33.216827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:33.217928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:33.217928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:33.217928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:33.217928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:33.217928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:33.217928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:33.217928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:33.217928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:33.217928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:33.217928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:33.217928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:33.217928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:33.217928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:33.217928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:33.217928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:33.217928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:33.218876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:33.218876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:33.218876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:33.218876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:33.218876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:33.218876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:33.218876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:33.218876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:33.218876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:33.218876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:33.218876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 14:26:33.218876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:33.218876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 161/307] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:33.218876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03f3de-761a-4e86-9e61-2274863f4239 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:33.290006 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:33.297604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 147/308] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 14:26:33.298248 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed952e-2108-41ea-af5c-dd3e5edbe0b6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:33.353669 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03f3de-761a-4e86-9e61-2274863f4239 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:33.368764 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:33.406087 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19965aa-6e73-454f-b4f6-68458b73f564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:33.406706 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19965aa-6e73-454f-b4f6-68458b73f564 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:33.407014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19965aa-6e73-454f-b4f6-68458b73f564 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:33.407435 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19965aa-6e73-454f-b4f6-68458b73f564 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:33.434632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed952e-2108-41ea-af5c-dd3e5edbe0b6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:33.476020 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75843ec-8f0d-4169-aace-e9b580a8e30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:33.476628 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75843ec-8f0d-4169-aace-e9b580a8e30e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:33.476904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75843ec-8f0d-4169-aace-e9b580a8e30e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:33.477172 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75843ec-8f0d-4169-aace-e9b580a8e30e None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:33.477840 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75843ec-8f0d-4169-aace-e9b580a8e30e None admin] RBAC: Adding query filter params (name=member) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:33.478139 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75843ec-8f0d-4169-aace-e9b580a8e30e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:33.479541 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75843ec-8f0d-4169-aace-e9b580a8e30e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:33.487987 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 148/309] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:26:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:33.489301 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75843ec-8f0d-4169-aace-e9b580a8e30e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:33.498270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19965aa-6e73-454f-b4f6-68458b73f564 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:33.499790 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19965aa-6e73-454f-b4f6-68458b73f564 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:33.564177 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:33.625573 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75843ec-8f0d-4169-aace-e9b580a8e30e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:33.635599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 162/310] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 14:26:33.637081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f118e2-27f3-431c-96b2-a5aa1cae3b94 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:33.637506 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f118e2-27f3-431c-96b2-a5aa1cae3b94 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:33.637921 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f118e2-27f3-431c-96b2-a5aa1cae3b94 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:33.639564 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 163/311] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:33.644353 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f118e2-27f3-431c-96b2-a5aa1cae3b94 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:33.663869 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:33.675998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e42c9-b55c-4e59-90bb-c55d2faf065e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:33.676645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e42c9-b55c-4e59-90bb-c55d2faf065e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:33.676970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e42c9-b55c-4e59-90bb-c55d2faf065e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:33.677250 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e42c9-b55c-4e59-90bb-c55d2faf065e None admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:33.678433 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e42c9-b55c-4e59-90bb-c55d2faf065e None admin] RBAC: Authorizing `identity:get_project(project_id=ad7b7e0473f64a4685d43fc69df8b643)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:33.680327 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e42c9-b55c-4e59-90bb-c55d2faf065e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:33.682254 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 149/312] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:33 2022] GET /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:33.682940 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e42c9-b55c-4e59-90bb-c55d2faf065e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:33.784344 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f118e2-27f3-431c-96b2-a5aa1cae3b94 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:33.811651 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e42c9-b55c-4e59-90bb-c55d2faf065e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:33.826541 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3a9ce-10ae-4d52-bf8d-1f82b532fa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:33.827073 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3a9ce-10ae-4d52-bf8d-1f82b532fa22 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:33.827391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3a9ce-10ae-4d52-bf8d-1f82b532fa22 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:33.827658 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3a9ce-10ae-4d52-bf8d-1f82b532fa22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:33.845628 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:33.860171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb3e19-841f-48f1-98e6-26553585e94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:33.861084 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb3e19-841f-48f1-98e6-26553585e94c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:33.861372 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb3e19-841f-48f1-98e6-26553585e94c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:33.861750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb3e19-841f-48f1-98e6-26553585e94c None admin] PATH_INFO: `/v3/projects/ad7b7e0473f64a4685d43fc69df8b643` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:33.862975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb3e19-841f-48f1-98e6-26553585e94c None admin] RBAC: Authorizing `identity:get_project(project_id=ad7b7e0473f64a4685d43fc69df8b643)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:33.867305 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb3e19-841f-48f1-98e6-26553585e94c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:33.867305 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 150/313] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:33 2022] GET /identity/v3/projects/ad7b7e0473f64a4685d43fc69df8b643 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:33.867305 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdda2f6-4d60-4476-b766-d9bb9b4cbcb1 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:33.867665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdda2f6-4d60-4476-b766-d9bb9b4cbcb1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:33.867878 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdda2f6-4d60-4476-b766-d9bb9b4cbcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:33.890119 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-5fdda2f6-4d60-4476-b766-d9bb9b4cbcb1 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:33.891436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-5fdda2f6-4d60-4476-b766-d9bb9b4cbcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:33.907209 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3a9ce-10ae-4d52-bf8d-1f82b532fa22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:33.909121 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3a9ce-10ae-4d52-bf8d-1f82b532fa22 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:33.925039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:33.982911 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 151/314] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:33.983886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdda2f6-4d60-4476-b766-d9bb9b4cbcb1 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:34.030859 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 164/315] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 14:26:34.031737 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3a9ce-10ae-4d52-bf8d-1f82b532fa22 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:34.108256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdda2f6-4d60-4476-b766-d9bb9b4cbcb1 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:34.154115 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:34.159850 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979825d2-736b-41f3-b1c3-c45da27de5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:34.160503 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979825d2-736b-41f3-b1c3-c45da27de5dd None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:34.160780 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979825d2-736b-41f3-b1c3-c45da27de5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:34.161051 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979825d2-736b-41f3-b1c3-c45da27de5dd None admin] PATH_INFO: `/v3/groups/8e91e75a55854d5bb3ad03309346b75b` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:34.162994 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979825d2-736b-41f3-b1c3-c45da27de5dd None admin] RBAC: Authorizing `identity:get_group(group_id=8e91e75a55854d5bb3ad03309346b75b)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:34.164932 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979825d2-736b-41f3-b1c3-c45da27de5dd None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:34.171166 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 152/316] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Oct 12 14:26:33 2022] GET /identity/v3/groups/8e91e75a55854d5bb3ad03309346b75b => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:34.171166 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979825d2-736b-41f3-b1c3-c45da27de5dd None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:34.181193 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3a9ce-10ae-4d52-bf8d-1f82b532fa22 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:34.199863 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:34.229016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c9167-dd84-4c42-a656-02e03e9817b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:34.230332 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c9167-dd84-4c42-a656-02e03e9817b1 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:34.230958 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c9167-dd84-4c42-a656-02e03e9817b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:34.231462 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c9167-dd84-4c42-a656-02e03e9817b1 None admin] PATH_INFO: `/v3/groups/8e91e75a55854d5bb3ad03309346b75b` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:34.232940 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c9167-dd84-4c42-a656-02e03e9817b1 None admin] RBAC: Authorizing `identity:get_group(group_id=8e91e75a55854d5bb3ad03309346b75b)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:34.234798 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c9167-dd84-4c42-a656-02e03e9817b1 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:34.237494 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 165/317] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Oct 12 14:26:34 2022] GET /identity/v3/groups/8e91e75a55854d5bb3ad03309346b75b => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:34.238619 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c9167-dd84-4c42-a656-02e03e9817b1 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:34.302943 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979825d2-736b-41f3-b1c3-c45da27de5dd None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:34.352015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cfe58-ba11-4093-995f-4306e964b163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:34.352803 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cfe58-ba11-4093-995f-4306e964b163 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:34.353171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cfe58-ba11-4093-995f-4306e964b163 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:34.353536 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cfe58-ba11-4093-995f-4306e964b163 None admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:34.355257 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654cfe58-ba11-4093-995f-4306e964b163 None admin] RBAC: Authorizing `identity:get_project(project_id=a78b76de457545f7927d98a7a89f4cec)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:34.357161 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654cfe58-ba11-4093-995f-4306e964b163 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:34.359202 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 153/318] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:34 2022] GET /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:34.360635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cfe58-ba11-4093-995f-4306e964b163 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:34.387474 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c9167-dd84-4c42-a656-02e03e9817b1 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:34.433288 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:34.435547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede84c8-8cfe-4442-b1d1-bb223cc04205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:34.436186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede84c8-8cfe-4442-b1d1-bb223cc04205 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:34.436463 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede84c8-8cfe-4442-b1d1-bb223cc04205 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:34.437165 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede84c8-8cfe-4442-b1d1-bb223cc04205 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:34.438215 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede84c8-8cfe-4442-b1d1-bb223cc04205 None admin] RBAC: Adding query filter params (group.id=8e91e75a55854d5bb3ad03309346b75b, role.id=6d1894444bfd400a89591929bd8925de, scope.project.id=ad7b7e0473f64a4685d43fc69df8b643) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:34.438785 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede84c8-8cfe-4442-b1d1-bb223cc04205 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:34.440286 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede84c8-8cfe-4442-b1d1-bb223cc04205 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:34.449158 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 166/319] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Oct 12 14:26:34 2022] GET /identity/v3/role_assignments?group.id=8e91e75a55854d5bb3ad03309346b75b&scope.project.id=ad7b7e0473f64a4685d43fc69df8b643&role.id=6d1894444bfd400a89591929bd8925de => generated 611 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:34.449993 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede84c8-8cfe-4442-b1d1-bb223cc04205 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:34.484545 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cfe58-ba11-4093-995f-4306e964b163 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:34.518793 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:34.539576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b48c89-c152-4240-826f-89e276323446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:34.540465 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b48c89-c152-4240-826f-89e276323446 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:34.540465 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b48c89-c152-4240-826f-89e276323446 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:34.540730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b48c89-c152-4240-826f-89e276323446 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:34.541513 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b48c89-c152-4240-826f-89e276323446 None admin] RBAC: Adding query filter params (group.id=8e91e75a55854d5bb3ad03309346b75b, role.id=f0761dd38b794ee6bfab00ac01f240cd, scope.project.id=ad7b7e0473f64a4685d43fc69df8b643) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:34.541822 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b48c89-c152-4240-826f-89e276323446 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:34.543324 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b48c89-c152-4240-826f-89e276323446 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:34.556093 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 154/320] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Oct 12 14:26:34 2022] GET /identity/v3/role_assignments?group.id=8e91e75a55854d5bb3ad03309346b75b&scope.project.id=ad7b7e0473f64a4685d43fc69df8b643&role.id=f0761dd38b794ee6bfab00ac01f240cd => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:34.556978 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b48c89-c152-4240-826f-89e276323446 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:34.601950 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede84c8-8cfe-4442-b1d1-bb223cc04205 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:34.645294 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75176ed3-8891-47c5-bb1e-5f3f540095ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:34.646383 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75176ed3-8891-47c5-bb1e-5f3f540095ab None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:34.647465 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75176ed3-8891-47c5-bb1e-5f3f540095ab None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:34.648111 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75176ed3-8891-47c5-bb1e-5f3f540095ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:34.661553 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75176ed3-8891-47c5-bb1e-5f3f540095ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:34.667499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-75176ed3-8891-47c5-bb1e-5f3f540095ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:26:34.667499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:34.667499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:34.667499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:34.667499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:34.667499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:34.667499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:34.667499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:34.667499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:34.667499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:34.667499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:34.667499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:34.667499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:34.667499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:34.667499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:34.667499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:34.668420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:34.668420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:34.668420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:34.668420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:34.668420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:34.668420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:34.668420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:34.668420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:34.668420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:34.668420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:34.668420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:34.668420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:34.668420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:34.668420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:34.668420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:34.668420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:34.669180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:34.669180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:34.669180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:34.669180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:34.669180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:34.669180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:34.669180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:34.669180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:34.669180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:34.669180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:34.669180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:26:34.669180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:34.671887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 167/321] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Oct 12 14:26:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:34.677895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75176ed3-8891-47c5-bb1e-5f3f540095ab None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:34.762822 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b48c89-c152-4240-826f-89e276323446 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:34.794984 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:34.804498 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d8307-5079-4229-b6ab-e180a807d58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:34.805152 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d8307-5079-4229-b6ab-e180a807d58d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:34.805432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d8307-5079-4229-b6ab-e180a807d58d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:34.805705 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d8307-5079-4229-b6ab-e180a807d58d None admin] PATH_INFO: `/v3/groups/8e91e75a55854d5bb3ad03309346b75b` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:34.808261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d8307-5079-4229-b6ab-e180a807d58d None admin] RBAC: Authorizing `identity:get_group(group_id=8e91e75a55854d5bb3ad03309346b75b)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:34.810164 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d8307-5079-4229-b6ab-e180a807d58d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:34.812630 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 155/322] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Oct 12 14:26:34 2022] GET /identity/v3/groups/8e91e75a55854d5bb3ad03309346b75b => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:34.816141 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a0093-605f-4b94-9b16-148edc7cd8f6 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:34.816534 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a0093-605f-4b94-9b16-148edc7cd8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:34.816930 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a0093-605f-4b94-9b16-148edc7cd8f6 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:34.819343 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 156/323] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:34.820183 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a0093-605f-4b94-9b16-148edc7cd8f6 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:34.821626 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75176ed3-8891-47c5-bb1e-5f3f540095ab None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:34.826118 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:34.869353 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c0b29-62c8-43c8-b202-9c5609d147d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:34.870021 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c0b29-62c8-43c8-b202-9c5609d147d1 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:34.870294 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c0b29-62c8-43c8-b202-9c5609d147d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:34.870564 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c0b29-62c8-43c8-b202-9c5609d147d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:34.871310 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c0b29-62c8-43c8-b202-9c5609d147d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:34.871615 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c0b29-62c8-43c8-b202-9c5609d147d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:34.873011 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c0b29-62c8-43c8-b202-9c5609d147d1 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:34.885023 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 168/324] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:34.886623 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9000565-abf7-44cd-90fd-5613ae712577 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:34.887344 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9000565-abf7-44cd-90fd-5613ae712577 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:34.888158 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9000565-abf7-44cd-90fd-5613ae712577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:34.910074 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-a9000565-abf7-44cd-90fd-5613ae712577 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:34.911320 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-a9000565-abf7-44cd-90fd-5613ae712577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:34.954934 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a0093-605f-4b94-9b16-148edc7cd8f6 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:34.988604 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 169/325] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:34.989553 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9000565-abf7-44cd-90fd-5613ae712577 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:34.994007 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2a5f2-4a3e-4b11-aaf9-504e7748acca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:34.994794 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2a5f2-4a3e-4b11-aaf9-504e7748acca None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:34.995242 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2a5f2-4a3e-4b11-aaf9-504e7748acca None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:34.995654 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2a5f2-4a3e-4b11-aaf9-504e7748acca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:34.996540 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2a5f2-4a3e-4b11-aaf9-504e7748acca None admin] RBAC: Adding query filter params (group.id=8e91e75a55854d5bb3ad03309346b75b, role.id=6d1894444bfd400a89591929bd8925de, scope.project.id=a78b76de457545f7927d98a7a89f4cec) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:34.996988 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2a5f2-4a3e-4b11-aaf9-504e7748acca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:34.998715 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2a5f2-4a3e-4b11-aaf9-504e7748acca None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:35.008163 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 157/326] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Oct 12 14:26:34 2022] GET /identity/v3/role_assignments?group.id=8e91e75a55854d5bb3ad03309346b75b&scope.project.id=a78b76de457545f7927d98a7a89f4cec&role.id=6d1894444bfd400a89591929bd8925de => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:35.111372 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:35.118718 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9000565-abf7-44cd-90fd-5613ae712577 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:35.158761 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f8c7c-dd0a-4e5d-b02f-56c0c60edb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:35.159407 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f8c7c-dd0a-4e5d-b02f-56c0c60edb58 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:35.159682 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f8c7c-dd0a-4e5d-b02f-56c0c60edb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:35.159947 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f8c7c-dd0a-4e5d-b02f-56c0c60edb58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:35.235413 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f8c7c-dd0a-4e5d-b02f-56c0c60edb58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:35.237146 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f8c7c-dd0a-4e5d-b02f-56c0c60edb58 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:35.272564 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2a5f2-4a3e-4b11-aaf9-504e7748acca None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:35.336970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:35.339863 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:35.352107 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 170/327] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 14:26:35.357728 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f8c7c-dd0a-4e5d-b02f-56c0c60edb58 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:35.394185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2a5f2-4a3e-4b11-aaf9-504e7748acca None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:35.432493 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d46e26-7177-4758-b53f-33d6ef04af56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:35.433132 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d46e26-7177-4758-b53f-33d6ef04af56 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:35.433409 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d46e26-7177-4758-b53f-33d6ef04af56 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:35.433675 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d46e26-7177-4758-b53f-33d6ef04af56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:35.446125 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d46e26-7177-4758-b53f-33d6ef04af56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:35.451902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-77d46e26-7177-4758-b53f-33d6ef04af56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:26:35.451902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:35.451902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:35.451902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:35.451902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:35.451902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:35.451902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:35.451902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:35.451902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:35.451902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:35.451902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:35.451902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:35.451902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:35.451902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:35.451902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:35.451902 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:35.452761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:35.452761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:35.452761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:35.452761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:35.452761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:35.452761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:35.452761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:35.452761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:35.452761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:35.452761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:35.452761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:35.452761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:35.452761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:35.452761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:35.452761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:35.452761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:35.453518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:35.453518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:35.453518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:35.453518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:35.453518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:35.453518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:35.453518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:35.453518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:35.453518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:35.453518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:35.453518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:26:35.453518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:35.453518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 158/328] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:26:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:35.458744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d46e26-7177-4758-b53f-33d6ef04af56 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:35.474936 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f8c7c-dd0a-4e5d-b02f-56c0c60edb58 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:35.513909 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4ad1b-be35-4ab7-8def-549675c2d49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:35.514464 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4ad1b-be35-4ab7-8def-549675c2d49e None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:35.514740 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4ad1b-be35-4ab7-8def-549675c2d49e None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:35.515009 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4ad1b-be35-4ab7-8def-549675c2d49e None admin] PATH_INFO: `/v3/projects/a78b76de457545f7927d98a7a89f4cec` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:35.518822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4ad1b-be35-4ab7-8def-549675c2d49e None admin] RBAC: Authorizing `identity:get_project(project_id=a78b76de457545f7927d98a7a89f4cec)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:35.520358 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4ad1b-be35-4ab7-8def-549675c2d49e None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:35.522908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 171/329] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:35 2022] GET /identity/v3/projects/a78b76de457545f7927d98a7a89f4cec => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:35.528524 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4ad1b-be35-4ab7-8def-549675c2d49e None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:35.567819 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d46e26-7177-4758-b53f-33d6ef04af56 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:35.584239 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:35.593046 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:35.602477 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72494268-cb46-44d0-a420-7ad4758f1111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:35.603084 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72494268-cb46-44d0-a420-7ad4758f1111 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:35.603391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72494268-cb46-44d0-a420-7ad4758f1111 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:35.603663 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72494268-cb46-44d0-a420-7ad4758f1111 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:35.604325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72494268-cb46-44d0-a420-7ad4758f1111 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:35.604627 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72494268-cb46-44d0-a420-7ad4758f1111 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:35.606003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72494268-cb46-44d0-a420-7ad4758f1111 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:35.613905 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 159/330] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:35.619762 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72494268-cb46-44d0-a420-7ad4758f1111 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:35.631759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4ad1b-be35-4ab7-8def-549675c2d49e None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:35.666628 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c54afd-37bd-4f0a-9a4d-902a02e94972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:35.667309 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c54afd-37bd-4f0a-9a4d-902a02e94972 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:35.667586 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c54afd-37bd-4f0a-9a4d-902a02e94972 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:35.667854 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c54afd-37bd-4f0a-9a4d-902a02e94972 None admin] PATH_INFO: `/v3/groups/8e91e75a55854d5bb3ad03309346b75b` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:35.668957 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c54afd-37bd-4f0a-9a4d-902a02e94972 None admin] RBAC: Authorizing `identity:get_group(group_id=8e91e75a55854d5bb3ad03309346b75b)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:35.670376 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c54afd-37bd-4f0a-9a4d-902a02e94972 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:35.672166 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 172/331] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Oct 12 14:26:35 2022] GET /identity/v3/groups/8e91e75a55854d5bb3ad03309346b75b => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:35.677755 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c54afd-37bd-4f0a-9a4d-902a02e94972 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:35.718344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72494268-cb46-44d0-a420-7ad4758f1111 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:35.750593 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc80b2-d584-4a41-ab45-6e645efb5b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:35.751511 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc80b2-d584-4a41-ab45-6e645efb5b27 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:35.751791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc80b2-d584-4a41-ab45-6e645efb5b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:35.752063 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc80b2-d584-4a41-ab45-6e645efb5b27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:35.776794 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c54afd-37bd-4f0a-9a4d-902a02e94972 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:35.798877 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:35.805480 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:35.810559 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0be811-83e6-4a76-b31c-3b673392f7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:35.811257 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0be811-83e6-4a76-b31c-3b673392f7ca None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:35.811602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0be811-83e6-4a76-b31c-3b673392f7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:35.811948 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0be811-83e6-4a76-b31c-3b673392f7ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:35.812655 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0be811-83e6-4a76-b31c-3b673392f7ca None admin] RBAC: Adding query filter params (group.id=8e91e75a55854d5bb3ad03309346b75b, role.id=f0761dd38b794ee6bfab00ac01f240cd, scope.project.id=a78b76de457545f7927d98a7a89f4cec) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:35.813294 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0be811-83e6-4a76-b31c-3b673392f7ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:35.814722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0be811-83e6-4a76-b31c-3b673392f7ca None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:35.820912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 173/332] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Oct 12 14:26:35 2022] GET /identity/v3/role_assignments?group.id=8e91e75a55854d5bb3ad03309346b75b&scope.project.id=a78b76de457545f7927d98a7a89f4cec&role.id=f0761dd38b794ee6bfab00ac01f240cd => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:35.821446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc80b2-d584-4a41-ab45-6e645efb5b27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:35.822958 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc80b2-d584-4a41-ab45-6e645efb5b27 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:35.927607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 160/333] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 12 14:26:35.933444 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0be811-83e6-4a76-b31c-3b673392f7ca None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:36.038919 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0be811-83e6-4a76-b31c-3b673392f7ca None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:36.073511 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3963b84-fe66-4859-b818-5b9ae1c89b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:36.074140 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3963b84-fe66-4859-b818-5b9ae1c89b32 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:36.074414 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3963b84-fe66-4859-b818-5b9ae1c89b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:36.074683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3963b84-fe66-4859-b818-5b9ae1c89b32 None admin] PATH_INFO: `/v3/projects/671bb1b187584e1b95d55373780c0171` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:36.075797 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3963b84-fe66-4859-b818-5b9ae1c89b32 None admin] RBAC: Authorizing `identity:get_project(project_id=671bb1b187584e1b95d55373780c0171)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:36.077219 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3963b84-fe66-4859-b818-5b9ae1c89b32 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:36.078836 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 174/334] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:26:35 2022] GET /identity/v3/projects/671bb1b187584e1b95d55373780c0171 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:36.085199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc80b2-d584-4a41-ab45-6e645efb5b27 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:36.165117 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:36.187621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc80b2-d584-4a41-ab45-6e645efb5b27 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:36.220555 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1758c4-d632-46d6-aa4d-fcecf3441608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:36.221180 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1758c4-d632-46d6-aa4d-fcecf3441608 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:36.221455 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1758c4-d632-46d6-aa4d-fcecf3441608 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:36.221724 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1758c4-d632-46d6-aa4d-fcecf3441608 None admin] PATH_INFO: `/v3/groups/425459424c20415e9fc0bad2efd0958c` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:36.222814 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1758c4-d632-46d6-aa4d-fcecf3441608 None admin] RBAC: Authorizing `identity:get_group(group_id=425459424c20415e9fc0bad2efd0958c)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:36.224277 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1758c4-d632-46d6-aa4d-fcecf3441608 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:36.226702 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 161/335] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Oct 12 14:26:36 2022] GET /identity/v3/groups/425459424c20415e9fc0bad2efd0958c => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:36.232716 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3963b84-fe66-4859-b818-5b9ae1c89b32 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:36.295972 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:36.332283 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3963b84-fe66-4859-b818-5b9ae1c89b32 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:36.365225 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da5e6e-c703-4435-88f4-017700b90f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:36.366016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da5e6e-c703-4435-88f4-017700b90f3e None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:36.366235 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da5e6e-c703-4435-88f4-017700b90f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:36.366499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da5e6e-c703-4435-88f4-017700b90f3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:36.367235 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da5e6e-c703-4435-88f4-017700b90f3e None admin] RBAC: Adding query filter params (group.id=425459424c20415e9fc0bad2efd0958c, role.id=4b1f48ab911e4fadbc70164ae43d508b, scope.project.id=671bb1b187584e1b95d55373780c0171) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:36.367537 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da5e6e-c703-4435-88f4-017700b90f3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:36.368963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da5e6e-c703-4435-88f4-017700b90f3e None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:36.374386 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 175/336] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Oct 12 14:26:36 2022] GET /identity/v3/role_assignments?group.id=425459424c20415e9fc0bad2efd0958c&scope.project.id=671bb1b187584e1b95d55373780c0171&role.id=4b1f48ab911e4fadbc70164ae43d508b => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:38.841808 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea471b-e416-4199-bd51-c8c9989c95e7 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:38.842313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea471b-e416-4199-bd51-c8c9989c95e7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:38.842313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea471b-e416-4199-bd51-c8c9989c95e7 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:38.843861 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 162/337] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:38.849708 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86272d5-9937-41de-bf62-4d2dfab8637b None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:38.850192 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86272d5-9937-41de-bf62-4d2dfab8637b None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:38.850443 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86272d5-9937-41de-bf62-4d2dfab8637b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:38.876323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-d86272d5-9937-41de-bf62-4d2dfab8637b None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:38.877467 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-d86272d5-9937-41de-bf62-4d2dfab8637b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:38.973621 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 176/338] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:39.150888 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ba86b-aad2-4b4f-80a0-0d276bf25f7c None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:39.151240 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ba86b-aad2-4b4f-80a0-0d276bf25f7c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:39.151454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ba86b-aad2-4b4f-80a0-0d276bf25f7c None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:39.153051 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 163/339] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:39.158480 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e9b01-6a98-4982-8a4b-798a8436bf6c None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:39.158759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e9b01-6a98-4982-8a4b-798a8436bf6c None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:39.159025 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e9b01-6a98-4982-8a4b-798a8436bf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:39.184028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-a90e9b01-6a98-4982-8a4b-798a8436bf6c None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:39.185193 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-a90e9b01-6a98-4982-8a4b-798a8436bf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:39.258334 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:39.280751 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 177/340] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:39.327885 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ba86b-aad2-4b4f-80a0-0d276bf25f7c None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:39.446537 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46a906-cc13-45fb-ba29-3b4b1e51b749 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:39.446726 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46a906-cc13-45fb-ba29-3b4b1e51b749 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:39.446992 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46a906-cc13-45fb-ba29-3b4b1e51b749 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:39.448780 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 178/341] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:39.453798 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa653f5-e65f-4d55-b11c-837cf08497de None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:39.454080 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa653f5-e65f-4d55-b11c-837cf08497de None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:39.454349 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa653f5-e65f-4d55-b11c-837cf08497de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:39.479829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ba86b-aad2-4b4f-80a0-0d276bf25f7c None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:39.491371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-6aa653f5-e65f-4d55-b11c-837cf08497de None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:39.492637 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-6aa653f5-e65f-4d55-b11c-837cf08497de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:39.549584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bf313-f00e-4783-a420-c034a091c75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:39.550219 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bf313-f00e-4783-a420-c034a091c75c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:39.550495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bf313-f00e-4783-a420-c034a091c75c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:39.550769 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bf313-f00e-4783-a420-c034a091c75c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:39.582852 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9bf313-f00e-4783-a420-c034a091c75c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:39.584420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9bf313-f00e-4783-a420-c034a091c75c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:39.595683 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:39.611916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-5a9bf313-f00e-4783-a420-c034a091c75c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:39.611916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:39.611916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:39.611916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:39.611916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:39.611916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:39.611916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:39.611916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:39.611916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:39.611916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:39.611916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:39.611916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:39.611916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:39.611916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:39.611916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:39.611916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:39.612917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:39.614375 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:39.614375 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:39.614375 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:39.614375 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 164/342] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:39.618867 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bf313-f00e-4783-a420-c034a091c75c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:39.636092 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 179/343] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:26:39.637040 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa653f5-e65f-4d55-b11c-837cf08497de None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:39.742501 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa653f5-e65f-4d55-b11c-837cf08497de None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:39.777730 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae6196-f121-44fd-a1c5-3a19bcd056ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:39.778364 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae6196-f121-44fd-a1c5-3a19bcd056ae None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:39.778642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae6196-f121-44fd-a1c5-3a19bcd056ae None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:39.778913 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae6196-f121-44fd-a1c5-3a19bcd056ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:39.789803 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae6196-f121-44fd-a1c5-3a19bcd056ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:39.791317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae6196-f121-44fd-a1c5-3a19bcd056ae None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:39.809442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-50ae6196-f121-44fd-a1c5-3a19bcd056ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:39.809442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:39.809442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:39.809442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:39.809442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:39.809442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:39.809442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:39.809442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:39.809442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:39.809442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:39.809442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:39.809442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:39.809442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:39.809442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:39.809442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:39.809442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:39.810393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:39.811477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:39.811477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:39.811477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:39.811477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 180/344] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:39.811698 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bf313-f00e-4783-a420-c034a091c75c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:39.812404 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:39.824317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae6196-f121-44fd-a1c5-3a19bcd056ae None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:39.869602 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09be0e-bcb3-424c-970c-4de114e7c068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:39.870220 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09be0e-bcb3-424c-970c-4de114e7c068 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:39.870466 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09be0e-bcb3-424c-970c-4de114e7c068 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:39.870728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09be0e-bcb3-424c-970c-4de114e7c068 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:39.871390 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc09be0e-bcb3-424c-970c-4de114e7c068 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:39.871682 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc09be0e-bcb3-424c-970c-4de114e7c068 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:39.873031 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc09be0e-bcb3-424c-970c-4de114e7c068 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:39.894037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 165/345] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:39.899204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09be0e-bcb3-424c-970c-4de114e7c068 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:39.974479 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:39.988492 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae6196-f121-44fd-a1c5-3a19bcd056ae None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:40.016882 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09be0e-bcb3-424c-970c-4de114e7c068 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:40.027241 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed7f72-df82-4454-8dfe-dbd4a301d1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:40.027853 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed7f72-df82-4454-8dfe-dbd4a301d1e3 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:40.028126 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed7f72-df82-4454-8dfe-dbd4a301d1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:40.028392 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed7f72-df82-4454-8dfe-dbd4a301d1e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:40.029028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed7f72-df82-4454-8dfe-dbd4a301d1e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:40.029327 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed7f72-df82-4454-8dfe-dbd4a301d1e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:40.030697 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed7f72-df82-4454-8dfe-dbd4a301d1e3 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:40.043022 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 181/346] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:40.044328 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb07295-e5b9-42f8-a496-cd1aa55b49b7 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:40.044611 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb07295-e5b9-42f8-a496-cd1aa55b49b7 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:40.044879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb07295-e5b9-42f8-a496-cd1aa55b49b7 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:40.046219 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 182/347] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:40.050863 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb07295-e5b9-42f8-a496-cd1aa55b49b7 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:40.058035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4473a-8ff7-48dd-b2af-b5aa99289935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:40.058647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4473a-8ff7-48dd-b2af-b5aa99289935 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:40.058919 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4473a-8ff7-48dd-b2af-b5aa99289935 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:40.059218 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4473a-8ff7-48dd-b2af-b5aa99289935 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:40.059860 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4473a-8ff7-48dd-b2af-b5aa99289935 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:40.061274 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4473a-8ff7-48dd-b2af-b5aa99289935 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:40.085125 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 166/348] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:26:39 2022] POST /identity/v3/users => generated 246 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 14:26:40.086245 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4473a-8ff7-48dd-b2af-b5aa99289935 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:40.098390 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:40.210119 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb07295-e5b9-42f8-a496-cd1aa55b49b7 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:40.225480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4473a-8ff7-48dd-b2af-b5aa99289935 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:40.246485 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:40.260492 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabda1c-b253-4b43-9e15-410007b1b335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:40.261532 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabda1c-b253-4b43-9e15-410007b1b335 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:40.261807 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabda1c-b253-4b43-9e15-410007b1b335 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:40.262077 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabda1c-b253-4b43-9e15-410007b1b335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:40.274175 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e5ec5-0c20-4b5c-b71f-b84b93d51697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:40.274812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e5ec5-0c20-4b5c-b71f-b84b93d51697 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:40.276526 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e5ec5-0c20-4b5c-b71f-b84b93d51697 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:40.276526 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e5ec5-0c20-4b5c-b71f-b84b93d51697 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:40.276833 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabda1c-b253-4b43-9e15-410007b1b335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:40.277840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabda1c-b253-4b43-9e15-410007b1b335 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:40.278085 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e5ec5-0c20-4b5c-b71f-b84b93d51697 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:40.279596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e5ec5-0c20-4b5c-b71f-b84b93d51697 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:40.294324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-3cabda1c-b253-4b43-9e15-410007b1b335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:40.294324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:40.294324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:40.294324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:40.294324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:40.294324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:40.294324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:40.294324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:40.294324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:40.294324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:40.294324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:40.294324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:40.294324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:40.294324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:40.294324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:40.294324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:40.295477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:40.296393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:40.296393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:40.296393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:40.296393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 183/349] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:40.297060 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac568114-1932-4626-b814-34c79b543bff None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:40.297060 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac568114-1932-4626-b814-34c79b543bff None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:40.297224 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac568114-1932-4626-b814-34c79b543bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:40.309798 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 167/350] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:26:40 2022] POST /identity/v3/users => generated 248 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:26:40.310647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e5ec5-0c20-4b5c-b71f-b84b93d51697 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:40.325322 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-ac568114-1932-4626-b814-34c79b543bff None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:40.326545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-ac568114-1932-4626-b814-34c79b543bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:40.409255 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 184/351] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:40.458901 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e5ec5-0c20-4b5c-b71f-b84b93d51697 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:40.502996 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a96c45-b0d8-4221-b793-7481e777921c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:40.503688 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a96c45-b0d8-4221-b793-7481e777921c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:40.503969 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a96c45-b0d8-4221-b793-7481e777921c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:40.504242 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a96c45-b0d8-4221-b793-7481e777921c None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:40.504919 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a96c45-b0d8-4221-b793-7481e777921c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:40.505329 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a96c45-b0d8-4221-b793-7481e777921c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:40.506997 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a96c45-b0d8-4221-b793-7481e777921c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:40.529037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 168/352] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:40.534823 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac568114-1932-4626-b814-34c79b543bff None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:40.561991 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:40.634335 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7565e37-e1ec-4c9e-af98-592ad734e078 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:40.634660 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7565e37-e1ec-4c9e-af98-592ad734e078 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:40.634885 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7565e37-e1ec-4c9e-af98-592ad734e078 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:40.636505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 169/353] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:26:40.642331 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce39ceb-59c0-4d2f-af88-73a3d3465fde None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:40.642611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce39ceb-59c0-4d2f-af88-73a3d3465fde None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:40.642885 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce39ceb-59c0-4d2f-af88-73a3d3465fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:40.670786 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-7ce39ceb-59c0-4d2f-af88-73a3d3465fde None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:40.671704 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7ce39ceb-59c0-4d2f-af88-73a3d3465fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:40.679512 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac568114-1932-4626-b814-34c79b543bff None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:40.707098 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:40.731982 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a542082-7839-4594-bdf4-f612ba873195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:40.732603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a542082-7839-4594-bdf4-f612ba873195 None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:40.732878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a542082-7839-4594-bdf4-f612ba873195 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:40.733149 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a542082-7839-4594-bdf4-f612ba873195 None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:40.733803 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a542082-7839-4594-bdf4-f612ba873195 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:40.735280 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a542082-7839-4594-bdf4-f612ba873195 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:40.770009 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 170/354] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:26:40.771362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce39ceb-59c0-4d2f-af88-73a3d3465fde None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:40.781489 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 185/355] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:26:40 2022] POST /identity/v3/users => generated 248 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:26:40.887204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce39ceb-59c0-4d2f-af88-73a3d3465fde None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:40.925460 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47bda7-16c0-484d-80ca-e45788b75d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:40.926110 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47bda7-16c0-484d-80ca-e45788b75d5e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:40.926384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47bda7-16c0-484d-80ca-e45788b75d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:40.926653 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47bda7-16c0-484d-80ca-e45788b75d5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:40.938220 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47bda7-16c0-484d-80ca-e45788b75d5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:40.939711 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47bda7-16c0-484d-80ca-e45788b75d5e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:40.956349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-8f47bda7-16c0-484d-80ca-e45788b75d5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:40.956349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:40.956349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:40.956349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:40.956349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:40.956349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:40.956349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:40.956349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:40.956349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:40.956349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:40.956349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:40.956349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:40.956349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:40.956349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:40.956349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:40.956349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:40.957285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:40.958109 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:40.958109 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:40.958109 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:40.958109 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 171/356] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:40.964081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a542082-7839-4594-bdf4-f612ba873195 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:41.055125 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:41.093626 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a542082-7839-4594-bdf4-f612ba873195 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:41.105026 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47bda7-16c0-484d-80ca-e45788b75d5e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:41.123485 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:41.138101 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb4d4e-0fd2-46be-8b8d-cdbfd3e74c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:41.139700 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb4d4e-0fd2-46be-8b8d-cdbfd3e74c15 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:41.140148 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb4d4e-0fd2-46be-8b8d-cdbfd3e74c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:41.140832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb4d4e-0fd2-46be-8b8d-cdbfd3e74c15 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:41.141623 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb4d4e-0fd2-46be-8b8d-cdbfd3e74c15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:41.142292 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb4d4e-0fd2-46be-8b8d-cdbfd3e74c15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:41.144534 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb4d4e-0fd2-46be-8b8d-cdbfd3e74c15 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:41.162981 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 186/357] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:41.167888 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb4d4e-0fd2-46be-8b8d-cdbfd3e74c15 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:41.229744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47bda7-16c0-484d-80ca-e45788b75d5e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:41.270476 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e315cf0-3dd9-4e4f-b661-90dfcd84d6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:41.271909 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e315cf0-3dd9-4e4f-b661-90dfcd84d6c8 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:41.272187 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e315cf0-3dd9-4e4f-b661-90dfcd84d6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:41.272719 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e315cf0-3dd9-4e4f-b661-90dfcd84d6c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:41.273393 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e315cf0-3dd9-4e4f-b661-90dfcd84d6c8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:41.274758 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e315cf0-3dd9-4e4f-b661-90dfcd84d6c8 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:41.283526 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.sql.core [None req-7e315cf0-3dd9-4e4f-b661-90dfcd84d6c8 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 12 14:26:41.283526 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 12 14:26:41.283526 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [parameters: {'id': 'aee7d769bfaa4a2e9b21ecf15d77c4f1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 12 14:26:41.283526 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81565) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 14:26:41.291263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-7e315cf0-3dd9-4e4f-b661-90dfcd84d6c8 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 12 14:26:41.291263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:41.291263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 14:26:41.291263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 14:26:41.291263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 14:26:41.291263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 14:26:41.291263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 14:26:41.291263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 14:26:41.291263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 14:26:41.291263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application conn.query(q) Oct 12 14:26:41.291263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 14:26:41.291263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 14:26:41.291263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 14:26:41.291263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result.read() Oct 12 14:26:41.291263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return ref.to_dict() Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 14:26:41.292136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application next(self.gen) Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application yield resource Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application next(self.gen) Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise self.value Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application session.commit() Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 14:26:41.293015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.session.flush() Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application compat.raise_( Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 14:26:41.293842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 14:26:41.294730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 14:26:41.294730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 14:26:41.294730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 12 14:26:41.294730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 12 14:26:41.294730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 14:26:41.294730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 14:26:41.294730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 14:26:41.294730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 14:26:41.294730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 14:26:41.294730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 14:26:41.294730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 14:26:41.294730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 14:26:41.294730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 14:26:41.294730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 14:26:41.294730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 14:26:41.294730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception Oct 12 14:26:41.295581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 14:26:41.295581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 14:26:41.295581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 14:26:41.295581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 14:26:41.295581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 14:26:41.295581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 14:26:41.295581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 14:26:41.295581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application conn.query(q) Oct 12 14:26:41.295581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 14:26:41.295581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 14:26:41.295581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 14:26:41.295581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result.read() Oct 12 14:26:41.295581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 14:26:41.295581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 14:26:41.295581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 14:26:41.295581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 14:26:41.296325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 14:26:41.296325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 14:26:41.296325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 14:26:41.296325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 14:26:41.296325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 12 14:26:41.296325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 12 14:26:41.296325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [parameters: {'id': 'aee7d769bfaa4a2e9b21ecf15d77c4f1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 12 14:26:41.296325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 14:26:41.296325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:41.296325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:26:41.296325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:41.296325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:41.296325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:41.296325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:41.296325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:41.296325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:41.297104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:41.297104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:41.297104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:41.297104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:41.297104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:41.297104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:41.297104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 12 14:26:41.297104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 12 14:26:41.297104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:41.297104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:41.297104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 12 14:26:41.297104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 12 14:26:41.297104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 14:26:41.297104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 14:26:41.297104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 12 14:26:41.297104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:41.300611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 172/358] 10.222.0.16 () {60 vars in 1131 bytes} [Wed Oct 12 14:26:41 2022] POST /identity/v3/roles => generated 144 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:41.308636 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e315cf0-3dd9-4e4f-b661-90dfcd84d6c8 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:41.325184 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb4d4e-0fd2-46be-8b8d-cdbfd3e74c15 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:41.375697 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:41.392186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94619d-f23d-405a-bee0-5bd6b07edaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:41.392863 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94619d-f23d-405a-bee0-5bd6b07edaef None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:41.393144 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94619d-f23d-405a-bee0-5bd6b07edaef None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:41.393418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94619d-f23d-405a-bee0-5bd6b07edaef None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:41.398523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c94619d-f23d-405a-bee0-5bd6b07edaef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:41.400025 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c94619d-f23d-405a-bee0-5bd6b07edaef None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:41.441985 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 187/359] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:26:41 2022] POST /identity/v3/users => generated 249 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 14:26:41.461279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:41.488862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e315cf0-3dd9-4e4f-b661-90dfcd84d6c8 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:41.527413 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c57ebd-fd92-4ac9-bbca-2fa54bed217d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:41.528110 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c57ebd-fd92-4ac9-bbca-2fa54bed217d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:41.528445 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c57ebd-fd92-4ac9-bbca-2fa54bed217d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:41.528723 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c57ebd-fd92-4ac9-bbca-2fa54bed217d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:41.540089 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c57ebd-fd92-4ac9-bbca-2fa54bed217d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:41.545983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-01c57ebd-fd92-4ac9-bbca-2fa54bed217d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:26:41.545983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:41.545983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:41.545983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:41.545983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:41.545983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:41.545983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:41.545983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:41.545983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:41.545983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:41.545983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:41.545983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:41.545983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:41.545983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:41.545983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:41.545983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:41.546974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:41.546974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:41.546974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:41.546974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:41.546974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:41.546974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:41.546974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:41.546974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:41.546974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:41.546974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:41.546974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:41.546974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:41.546974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:41.546974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:41.546974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:41.546974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:41.547829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:41.547829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:41.547829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:41.547829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:41.547829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:41.547829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:41.547829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:41.547829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:41.547829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:41.547829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:41.547829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 14:26:41.547829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:41.547829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 173/360] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Oct 12 14:26:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:41.554238 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94619d-f23d-405a-bee0-5bd6b07edaef None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:41.709853 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94619d-f23d-405a-bee0-5bd6b07edaef None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:41.747429 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede8301-6c4e-4504-bdea-4723043ca365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:41.748400 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede8301-6c4e-4504-bdea-4723043ca365 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:41.748874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede8301-6c4e-4504-bdea-4723043ca365 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:41.749338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede8301-6c4e-4504-bdea-4723043ca365 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:41.750198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede8301-6c4e-4504-bdea-4723043ca365 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:41.750721 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede8301-6c4e-4504-bdea-4723043ca365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:41.752362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede8301-6c4e-4504-bdea-4723043ca365 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:41.763507 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 188/361] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:42.632734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184aa3b-5a26-44c1-9a32-36d928818ad3 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:42.633269 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184aa3b-5a26-44c1-9a32-36d928818ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:42.633269 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184aa3b-5a26-44c1-9a32-36d928818ad3 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:42.634608 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 174/362] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:42.639911 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3c2f2-e69a-42ee-afd1-2c19054c5746 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:42.640304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3c2f2-e69a-42ee-afd1-2c19054c5746 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:42.640467 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3c2f2-e69a-42ee-afd1-2c19054c5746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:42.659480 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-28f3c2f2-e69a-42ee-afd1-2c19054c5746 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:42.660664 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-28f3c2f2-e69a-42ee-afd1-2c19054c5746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:42.719530 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:42.780278 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 189/363] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:43.187166 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb469121-bed4-4700-9f16-6b7af9968b41 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:43.187642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb469121-bed4-4700-9f16-6b7af9968b41 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:43.187818 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb469121-bed4-4700-9f16-6b7af9968b41 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:43.189280 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 175/364] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:43.197304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aead314-627e-4b9e-a813-be675c6a2d2a None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:43.198102 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aead314-627e-4b9e-a813-be675c6a2d2a None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:43.198829 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aead314-627e-4b9e-a813-be675c6a2d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:43.234982 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-1aead314-627e-4b9e-a813-be675c6a2d2a None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:43.237448 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-1aead314-627e-4b9e-a813-be675c6a2d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:43.376576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb469121-bed4-4700-9f16-6b7af9968b41 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:43.379192 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 190/365] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:43.541726 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb469121-bed4-4700-9f16-6b7af9968b41 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:43.591942 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e2aa4-7a85-4cd1-8701-bff3e9f33b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:43.592591 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e2aa4-7a85-4cd1-8701-bff3e9f33b60 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:43.592870 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e2aa4-7a85-4cd1-8701-bff3e9f33b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:43.593127 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e2aa4-7a85-4cd1-8701-bff3e9f33b60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:43.602122 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:43.626764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e2aa4-7a85-4cd1-8701-bff3e9f33b60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:43.637673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-d58e2aa4-7a85-4cd1-8701-bff3e9f33b60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:43.637673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:43.637673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:43.637673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:43.637673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:43.637673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:43.637673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:43.637673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:43.637673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:43.637673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:43.637673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:43.637673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:43.637673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:43.637673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:43.637673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:43.637673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:43.640101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:43.640101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:43.640101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:43.640101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:43.640101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:43.640101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:43.640101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:43.640101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:43.640101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:43.640101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:43.640101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:43.640101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:43.640101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:43.640101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:43.640101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:43.640101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:43.640975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:43.640975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:43.640975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:43.640975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:43.640975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:43.640975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:43.640975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:43.640975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:43.640975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:43.640975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:43.640975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:43.640975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:43.640975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 176/366] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Oct 12 14:26:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:43.649012 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aead314-627e-4b9e-a813-be675c6a2d2a None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:43.728314 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e2aa4-7a85-4cd1-8701-bff3e9f33b60 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:43.841683 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e2aa4-7a85-4cd1-8701-bff3e9f33b60 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:43.850797 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aead314-627e-4b9e-a813-be675c6a2d2a None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:43.866255 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:43.885207 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70f50c-d802-4888-b369-237d5cc01710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:43.886296 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70f50c-d802-4888-b369-237d5cc01710 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:43.886779 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70f50c-d802-4888-b369-237d5cc01710 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:43.887279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70f50c-d802-4888-b369-237d5cc01710 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:43.897514 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e88dc-9147-4cd8-8b1c-fa040990a9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:43.898119 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e88dc-9147-4cd8-8b1c-fa040990a9c6 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:43.898388 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e88dc-9147-4cd8-8b1c-fa040990a9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:43.898651 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e88dc-9147-4cd8-8b1c-fa040990a9c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:43.899340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e88dc-9147-4cd8-8b1c-fa040990a9c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:43.899636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e88dc-9147-4cd8-8b1c-fa040990a9c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:43.900998 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e88dc-9147-4cd8-8b1c-fa040990a9c6 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:43.901542 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70f50c-d802-4888-b369-237d5cc01710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:43.908835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-3c70f50c-d802-4888-b369-237d5cc01710 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:43.908835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:43.908835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:43.908835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:43.908835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:43.908835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:43.908835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:43.908835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:43.908835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:43.908835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:43.908835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:43.908835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:43.908835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:43.908835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:43.908835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:43.908835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:43.909897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:43.909897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:43.909897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:43.909897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:43.909897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:43.909897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:43.909897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:43.909897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:43.909897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:43.909897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:43.909897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:43.909897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:43.909897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:43.909897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:43.909897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:43.909897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:43.910678 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:43.910678 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:43.910678 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:43.910678 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:43.910678 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:43.910678 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:43.910678 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:43.910678 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:43.910678 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:43.910678 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:43.910678 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:43.910678 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:43.911583 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 191/367] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Oct 12 14:26:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:43.911583 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cc4d8-3933-4386-b662-403d57e90919 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:43.911583 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cc4d8-3933-4386-b662-403d57e90919 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:43.911583 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cc4d8-3933-4386-b662-403d57e90919 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:43.913742 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 177/368] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Oct 12 14:26:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:43.918807 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 192/369] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:43.919880 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70f50c-d802-4888-b369-237d5cc01710 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:43.921276 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84cc4d8-3933-4386-b662-403d57e90919 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:43.991123 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:44.039668 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84cc4d8-3933-4386-b662-403d57e90919 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:44.052678 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70f50c-d802-4888-b369-237d5cc01710 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:44.079054 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd746d91-47e8-4527-b60d-dc3c442f238a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:44.079753 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd746d91-47e8-4527-b60d-dc3c442f238a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:44.080441 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd746d91-47e8-4527-b60d-dc3c442f238a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:44.080441 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd746d91-47e8-4527-b60d-dc3c442f238a None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:44.081060 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd746d91-47e8-4527-b60d-dc3c442f238a None admin] RBAC: Adding query filter params (name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:44.081360 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd746d91-47e8-4527-b60d-dc3c442f238a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:44.082773 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd746d91-47e8-4527-b60d-dc3c442f238a None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:44.096724 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 193/370] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Oct 12 14:26:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:44.096724 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d3846-8416-45b5-9c3a-e7a03f68a841 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:44.096724 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d3846-8416-45b5-9c3a-e7a03f68a841 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:44.096724 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d3846-8416-45b5-9c3a-e7a03f68a841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:44.103792 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e98fa-00c1-40e2-bd21-58519985e9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:44.104713 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e98fa-00c1-40e2-bd21-58519985e9ac None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:44.105192 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e98fa-00c1-40e2-bd21-58519985e9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:44.105659 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e98fa-00c1-40e2-bd21-58519985e9ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:44.113457 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-830d3846-8416-45b5-9c3a-e7a03f68a841 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:44.115188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-830d3846-8416-45b5-9c3a-e7a03f68a841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:44.119536 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e98fa-00c1-40e2-bd21-58519985e9ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:44.121212 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e98fa-00c1-40e2-bd21-58519985e9ac None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:44.142319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-5a7e98fa-00c1-40e2-bd21-58519985e9ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:44.142319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:44.142319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:44.142319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:44.142319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:44.142319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:44.142319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:44.142319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:44.142319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:44.142319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:44.142319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:44.142319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:44.142319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:44.142319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:44.142319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:44.142319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:44.143299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:44.144127 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:44.144127 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:44.144127 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:44.151227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 178/371] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:44.151774 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e98fa-00c1-40e2-bd21-58519985e9ac None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:44.151974 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:44.201913 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 194/372] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:44.202758 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d3846-8416-45b5-9c3a-e7a03f68a841 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:44.281230 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e98fa-00c1-40e2-bd21-58519985e9ac None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:44.296672 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:44.322663 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1ccf0-e9dd-4590-8645-5b93eb73cb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:44.323894 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1ccf0-e9dd-4590-8645-5b93eb73cb0e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:44.324373 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1ccf0-e9dd-4590-8645-5b93eb73cb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:44.325034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1ccf0-e9dd-4590-8645-5b93eb73cb0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:44.338330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1ccf0-e9dd-4590-8645-5b93eb73cb0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:44.340947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1ccf0-e9dd-4590-8645-5b93eb73cb0e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:44.360456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-ecc1ccf0-e9dd-4590-8645-5b93eb73cb0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:44.360456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:44.360456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:44.360456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:44.360456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:44.360456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:44.360456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:44.360456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:44.360456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:44.360456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:44.360456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:44.360456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:44.360456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:44.360456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:44.360456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:44.360456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:44.361349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:44.362167 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:44.362167 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:44.362167 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:44.364564 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 179/373] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:44.371070 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1ccf0-e9dd-4590-8645-5b93eb73cb0e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:44.421220 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d3846-8416-45b5-9c3a-e7a03f68a841 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:44.490203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1ccf0-e9dd-4590-8645-5b93eb73cb0e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:44.494273 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca00c15-e376-4dfc-a659-0ab1a34dbd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:44.494885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca00c15-e376-4dfc-a659-0ab1a34dbd9a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:44.495168 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca00c15-e376-4dfc-a659-0ab1a34dbd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:44.495464 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca00c15-e376-4dfc-a659-0ab1a34dbd9a None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:44.496095 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca00c15-e376-4dfc-a659-0ab1a34dbd9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:44.496385 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca00c15-e376-4dfc-a659-0ab1a34dbd9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:44.497768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca00c15-e376-4dfc-a659-0ab1a34dbd9a None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:44.519742 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 195/374] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:44.526469 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca00c15-e376-4dfc-a659-0ab1a34dbd9a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:44.530508 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d36f61-1e1a-45f9-8161-7275e77d7c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:44.531402 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d36f61-1e1a-45f9-8161-7275e77d7c69 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:44.531906 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d36f61-1e1a-45f9-8161-7275e77d7c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:44.532384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d36f61-1e1a-45f9-8161-7275e77d7c69 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:44.533235 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d36f61-1e1a-45f9-8161-7275e77d7c69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:44.533732 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d36f61-1e1a-45f9-8161-7275e77d7c69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:44.536374 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d36f61-1e1a-45f9-8161-7275e77d7c69 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:44.550440 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 180/375] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:44.556938 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d36f61-1e1a-45f9-8161-7275e77d7c69 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:44.569160 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:44.586777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:44.680410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d36f61-1e1a-45f9-8161-7275e77d7c69 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:44.730133 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d68a87-c3bd-488a-988f-0d90ea27c7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:44.730868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d68a87-c3bd-488a-988f-0d90ea27c7dc None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:44.731162 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d68a87-c3bd-488a-988f-0d90ea27c7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:44.731551 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d68a87-c3bd-488a-988f-0d90ea27c7dc None admin] PATH_INFO: `/v3/users/glance` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:44.737808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca00c15-e376-4dfc-a659-0ab1a34dbd9a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:44.739745 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d68a87-c3bd-488a-988f-0d90ea27c7dc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:44.741637 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d68a87-c3bd-488a-988f-0d90ea27c7dc None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:44.760609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-75d68a87-c3bd-488a-988f-0d90ea27c7dc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 12 14:26:44.760609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:44.760609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:44.760609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:44.760609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:44.760609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:44.760609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:44.760609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:44.760609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:44.760609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:44.760609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:44.760609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:44.760609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:26:44.760609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:26:44.760609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:26:44.760609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:44.761748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:44.762703 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:44.762703 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:44.762703 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:26:44.762703 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:26:44.762703 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:26:44.762703 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:26:44.762703 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:26:44.762703 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:26:44.762703 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 12 14:26:44.762703 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:44.762703 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 181/376] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:44.763297 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5cf1c-28b2-4027-9f59-42045f5ea28e None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:44.763725 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5cf1c-28b2-4027-9f59-42045f5ea28e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:44.763830 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5cf1c-28b2-4027-9f59-42045f5ea28e None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:44.765245 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 182/377] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:44.767934 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5cf1c-28b2-4027-9f59-42045f5ea28e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:44.779019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22e3af-650d-4b42-a15c-193e7f905a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:44.780061 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22e3af-650d-4b42-a15c-193e7f905a33 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:44.780339 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22e3af-650d-4b42-a15c-193e7f905a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:44.780605 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22e3af-650d-4b42-a15c-193e7f905a33 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:44.789210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22e3af-650d-4b42-a15c-193e7f905a33 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:44.790638 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22e3af-650d-4b42-a15c-193e7f905a33 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:44.803067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-9e22e3af-650d-4b42-a15c-193e7f905a33 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 14:26:44.803067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:44.803067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:44.803067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:44.803067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:44.803067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:44.803067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:44.803067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:44.803067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:44.803067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:44.803067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:44.803067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:44.803067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:26:44.803067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:26:44.803067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:26:44.803067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:44.804128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:44.805016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:44.805016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:44.805016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:26:44.805016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:26:44.805016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:26:44.805016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:26:44.805016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:26:44.805016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:26:44.805016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 14:26:44.805016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:44.805016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 196/378] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Oct 12 14:26:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:44.805546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85ac6d-2d2e-40dd-9f7e-ef3ee4bac53e None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:44.805546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85ac6d-2d2e-40dd-9f7e-ef3ee4bac53e None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:44.806490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85ac6d-2d2e-40dd-9f7e-ef3ee4bac53e None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:44.807123 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 197/379] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:44.807686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85ac6d-2d2e-40dd-9f7e-ef3ee4bac53e None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:44.854323 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:44.873737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5cf1c-28b2-4027-9f59-42045f5ea28e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:44.888629 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:44.907863 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d74deb-66e4-4c78-9379-3ec220bf7586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:44.908489 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d74deb-66e4-4c78-9379-3ec220bf7586 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:44.908763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d74deb-66e4-4c78-9379-3ec220bf7586 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:44.909032 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d74deb-66e4-4c78-9379-3ec220bf7586 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:44.909972 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85ac6d-2d2e-40dd-9f7e-ef3ee4bac53e None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:44.919792 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d74deb-66e4-4c78-9379-3ec220bf7586 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:44.924634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-23d74deb-66e4-4c78-9379-3ec220bf7586 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:44.924634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:44.924634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:44.924634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:44.924634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:44.924634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:44.924634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:44.924634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:44.924634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:44.924634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:44.924634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:44.924634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:44.924634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:44.924634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:44.924634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:44.924634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:44.925504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:44.925504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:44.925504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:44.925504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:44.925504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:44.925504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:44.925504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:44.925504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:44.925504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:44.925504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:44.925504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:44.925504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:44.925504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:44.925504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:44.925504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:44.925504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:44.926339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:44.926339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:44.926339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:44.926339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:44.926339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:44.926339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:44.926339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:44.926339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:44.926339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:44.926339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:44.926339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:44.926339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:44.926339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 183/380] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Oct 12 14:26:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:44.927462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2a8d5-7d3e-4a14-b444-48de420fd6d4 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:44.927675 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2a8d5-7d3e-4a14-b444-48de420fd6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:44.927948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2a8d5-7d3e-4a14-b444-48de420fd6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:44.945411 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856cbae5-fb77-486c-b35d-c6bec003c28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:44.945582 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-44e2a8d5-7d3e-4a14-b444-48de420fd6d4 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:44.945935 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cbae5-fb77-486c-b35d-c6bec003c28f None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:44.946275 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cbae5-fb77-486c-b35d-c6bec003c28f None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:44.946546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cbae5-fb77-486c-b35d-c6bec003c28f None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:44.946637 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-44e2a8d5-7d3e-4a14-b444-48de420fd6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:44.947291 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856cbae5-fb77-486c-b35d-c6bec003c28f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:44.947590 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856cbae5-fb77-486c-b35d-c6bec003c28f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:44.949013 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856cbae5-fb77-486c-b35d-c6bec003c28f None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:44.985734 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 198/381] 10.222.0.16 () {56 vars in 1137 bytes} [Wed Oct 12 14:26:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:44.986543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856cbae5-fb77-486c-b35d-c6bec003c28f None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:45.012900 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 184/382] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:45.016413 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac7812-2a34-497e-ae70-79645dc29f6f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:45.016691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac7812-2a34-497e-ae70-79645dc29f6f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:45.016957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac7812-2a34-497e-ae70-79645dc29f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:45.034664 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9aac7812-2a34-497e-ae70-79645dc29f6f None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:45.036540 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9aac7812-2a34-497e-ae70-79645dc29f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:45.094565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856cbae5-fb77-486c-b35d-c6bec003c28f None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:45.101327 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:45.107206 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 185/383] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:45.107948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac7812-2a34-497e-ae70-79645dc29f6f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:45.146702 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0363e-c11b-40d6-a368-518d98764f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:45.147936 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0363e-c11b-40d6-a368-518d98764f0d None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:45.148406 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0363e-c11b-40d6-a368-518d98764f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:45.148878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0363e-c11b-40d6-a368-518d98764f0d None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:45.150341 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0363e-c11b-40d6-a368-518d98764f0d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:45.150855 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0363e-c11b-40d6-a368-518d98764f0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:45.152955 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0363e-c11b-40d6-a368-518d98764f0d None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:45.164766 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:45.179720 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 199/384] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Oct 12 14:26:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:45.181202 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0363e-c11b-40d6-a368-518d98764f0d None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:45.215628 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac7812-2a34-497e-ae70-79645dc29f6f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:45.259127 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202672ec-ee8f-47fe-b1b0-e5ad546cc620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:45.260240 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202672ec-ee8f-47fe-b1b0-e5ad546cc620 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:45.260657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202672ec-ee8f-47fe-b1b0-e5ad546cc620 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:45.261298 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202672ec-ee8f-47fe-b1b0-e5ad546cc620 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:45.262509 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202672ec-ee8f-47fe-b1b0-e5ad546cc620 None admin] RBAC: Adding query filter params (name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:45.263217 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202672ec-ee8f-47fe-b1b0-e5ad546cc620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:45.264948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202672ec-ee8f-47fe-b1b0-e5ad546cc620 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:45.277399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 186/385] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Oct 12 14:26:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:45.278382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202672ec-ee8f-47fe-b1b0-e5ad546cc620 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:45.306260 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0363e-c11b-40d6-a368-518d98764f0d None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:45.342011 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:45.345718 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20b26b-9614-4402-947e-1767f1d957da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:45.346671 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20b26b-9614-4402-947e-1767f1d957da None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:45.347093 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20b26b-9614-4402-947e-1767f1d957da None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:45.347524 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20b26b-9614-4402-947e-1767f1d957da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:45.357917 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20b26b-9614-4402-947e-1767f1d957da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:45.359594 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20b26b-9614-4402-947e-1767f1d957da None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:45.376912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-9e20b26b-9614-4402-947e-1767f1d957da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:45.376912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:45.376912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:45.376912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:45.376912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:45.376912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:45.376912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:45.376912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:45.376912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:45.376912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:45.376912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:45.376912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:45.376912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:45.376912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:45.376912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:45.376912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:45.377771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:45.378600 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:45.378600 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:45.378600 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:45.378600 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 200/386] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:45.378812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20b26b-9614-4402-947e-1767f1d957da None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:45.385612 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202672ec-ee8f-47fe-b1b0-e5ad546cc620 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:45.420843 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3edfdc-990a-4457-b9e0-dd0dfb9542b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:45.421461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3edfdc-990a-4457-b9e0-dd0dfb9542b0 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:45.421737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3edfdc-990a-4457-b9e0-dd0dfb9542b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:45.422007 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3edfdc-990a-4457-b9e0-dd0dfb9542b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:45.432446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3edfdc-990a-4457-b9e0-dd0dfb9542b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:45.433080 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:45.433919 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3edfdc-990a-4457-b9e0-dd0dfb9542b0 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:45.448315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-ba3edfdc-990a-4457-b9e0-dd0dfb9542b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:45.448315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:45.448315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:45.448315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:45.448315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:45.448315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:45.448315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:45.448315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:45.448315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:45.448315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:45.448315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:45.448315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:45.448315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:45.448315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:45.448315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:45.448315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:45.449199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:45.450012 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:45.450012 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:45.450012 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:45.450012 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 187/387] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:45.450219 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3edfdc-990a-4457-b9e0-dd0dfb9542b0 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:45.484604 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20b26b-9614-4402-947e-1767f1d957da None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:45.516834 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793eeac9-f649-40e0-85bc-4781864d1ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:45.517557 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793eeac9-f649-40e0-85bc-4781864d1ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:45.517726 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793eeac9-f649-40e0-85bc-4781864d1ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:45.517992 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793eeac9-f649-40e0-85bc-4781864d1ef0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:45.528788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793eeac9-f649-40e0-85bc-4781864d1ef0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:45.530222 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793eeac9-f649-40e0-85bc-4781864d1ef0 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:45.544715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-793eeac9-f649-40e0-85bc-4781864d1ef0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:45.544715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:45.544715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:45.544715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:45.544715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:45.544715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:45.544715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:45.544715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:45.544715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:45.544715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:45.544715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:45.544715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:45.544715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:45.544715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:45.544715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:45.544715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:45.545741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:45.550589 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:45.550589 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:45.550589 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:45.550589 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 201/388] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:45.550589 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793eeac9-f649-40e0-85bc-4781864d1ef0 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:45.560581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3edfdc-990a-4457-b9e0-dd0dfb9542b0 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:45.575623 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:45.600605 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37280189-4c69-4808-af06-d01bdf841457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:45.601231 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37280189-4c69-4808-af06-d01bdf841457 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:45.601506 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37280189-4c69-4808-af06-d01bdf841457 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:45.601774 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37280189-4c69-4808-af06-d01bdf841457 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:45.612025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37280189-4c69-4808-af06-d01bdf841457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:45.613465 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37280189-4c69-4808-af06-d01bdf841457 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:45.627483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-37280189-4c69-4808-af06-d01bdf841457 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:45.627483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:45.627483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:45.627483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:45.627483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:45.627483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:45.627483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:45.627483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:45.627483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:45.627483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:45.627483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:45.627483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:45.627483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:45.627483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:45.627483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:45.627483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:45.628410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:45.629713 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:45.629713 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:45.629713 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:45.629713 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 188/389] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:45.629931 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37280189-4c69-4808-af06-d01bdf841457 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:45.649735 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:45.652673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793eeac9-f649-40e0-85bc-4781864d1ef0 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:45.687555 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38081a-e477-48f6-ba7b-f6e565e69423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:45.688152 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38081a-e477-48f6-ba7b-f6e565e69423 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:45.688426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38081a-e477-48f6-ba7b-f6e565e69423 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:45.688698 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38081a-e477-48f6-ba7b-f6e565e69423 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:45.689323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38081a-e477-48f6-ba7b-f6e565e69423 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:45.689619 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38081a-e477-48f6-ba7b-f6e565e69423 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:45.690994 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38081a-e477-48f6-ba7b-f6e565e69423 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:45.702890 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 202/390] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:45.703699 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38081a-e477-48f6-ba7b-f6e565e69423 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:45.727345 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37280189-4c69-4808-af06-d01bdf841457 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:45.761188 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94096470-25f4-4e0a-a604-602db4014896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:45.762005 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94096470-25f4-4e0a-a604-602db4014896 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:45.762285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94096470-25f4-4e0a-a604-602db4014896 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:45.762556 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94096470-25f4-4e0a-a604-602db4014896 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:45.772790 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94096470-25f4-4e0a-a604-602db4014896 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:45.778021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-94096470-25f4-4e0a-a604-602db4014896 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:45.778021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:45.778021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:45.778021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:45.778021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:45.778021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:45.778021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:45.778021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:45.778021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:45.778021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:45.778021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:45.778021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:45.778021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:45.778021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:45.778021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:45.778021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:45.778914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:45.778914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:45.778914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:45.778914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:45.778914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:45.778914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:45.778914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:45.778914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:45.778914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:45.778914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:45.778914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:45.778914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:45.778914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:45.778914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:45.778914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:45.778914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:45.779970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:45.779970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:45.779970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:45.779970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:45.779970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:45.779970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:45.779970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:45.779970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:45.779970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:45.779970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:45.779970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:45.779970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:45.779970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 189/391] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Oct 12 14:26:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:45.779970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94096470-25f4-4e0a-a604-602db4014896 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:45.803879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38081a-e477-48f6-ba7b-f6e565e69423 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:45.806750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:45.837721 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b0821-23ee-44a0-bca6-e385404ee3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:45.838321 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b0821-23ee-44a0-bca6-e385404ee3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:45.838594 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b0821-23ee-44a0-bca6-e385404ee3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:45.838862 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b0821-23ee-44a0-bca6-e385404ee3f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:45.839542 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b0821-23ee-44a0-bca6-e385404ee3f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:45.839841 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b0821-23ee-44a0-bca6-e385404ee3f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:45.841402 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b0821-23ee-44a0-bca6-e385404ee3f6 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:45.852176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 203/392] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:45.852950 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b0821-23ee-44a0-bca6-e385404ee3f6 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:45.874258 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:45.882692 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94096470-25f4-4e0a-a604-602db4014896 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:45.915707 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99889d2d-0598-4682-8eb1-209d4092284a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:45.916306 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99889d2d-0598-4682-8eb1-209d4092284a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:45.916584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99889d2d-0598-4682-8eb1-209d4092284a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:45.916857 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99889d2d-0598-4682-8eb1-209d4092284a None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:45.917492 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99889d2d-0598-4682-8eb1-209d4092284a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:45.917795 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99889d2d-0598-4682-8eb1-209d4092284a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:45.919172 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99889d2d-0598-4682-8eb1-209d4092284a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:45.931251 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 190/393] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:45.932043 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99889d2d-0598-4682-8eb1-209d4092284a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:45.955494 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b0821-23ee-44a0-bca6-e385404ee3f6 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:45.987977 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbce6a-1434-42e4-bac3-5c8cd092bbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:45.989190 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbce6a-1434-42e4-bac3-5c8cd092bbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:45.989514 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbce6a-1434-42e4-bac3-5c8cd092bbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:45.989876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbce6a-1434-42e4-bac3-5c8cd092bbf9 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:45.990505 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bbce6a-1434-42e4-bac3-5c8cd092bbf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:45.990801 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bbce6a-1434-42e4-bac3-5c8cd092bbf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:45.992216 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bbce6a-1434-42e4-bac3-5c8cd092bbf9 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:46.004203 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 204/394] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:46.005079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbce6a-1434-42e4-bac3-5c8cd092bbf9 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:46.019106 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:46.034772 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99889d2d-0598-4682-8eb1-209d4092284a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:46.068068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f0a51-a08e-421c-a67f-a2c566230d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:46.068696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f0a51-a08e-421c-a67f-a2c566230d59 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:46.068973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f0a51-a08e-421c-a67f-a2c566230d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:46.069245 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f0a51-a08e-421c-a67f-a2c566230d59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:46.075851 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f0a51-a08e-421c-a67f-a2c566230d59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:46.077298 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f0a51-a08e-421c-a67f-a2c566230d59 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:46.085697 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:46.089650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-cf1f0a51-a08e-421c-a67f-a2c566230d59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:46.089650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:46.089650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:46.089650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:46.089650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:46.089650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:46.089650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:46.089650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:46.089650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:46.089650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:46.089650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:46.089650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:46.089650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:26:46.089650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:26:46.089650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:46.090665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:46.091577 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:26:46.091577 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:26:46.091577 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:26:46.091577 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:26:46.091577 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:26:46.091577 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:26:46.091577 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:46.091577 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:46.091577 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 191/395] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:46.091577 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f0a51-a08e-421c-a67f-a2c566230d59 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:46.107902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbce6a-1434-42e4-bac3-5c8cd092bbf9 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:46.140788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09d77a-0a76-4fcc-8024-be92989c5863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:46.141380 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09d77a-0a76-4fcc-8024-be92989c5863 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:46.141649 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09d77a-0a76-4fcc-8024-be92989c5863 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:46.142247 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09d77a-0a76-4fcc-8024-be92989c5863 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:46.142548 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09d77a-0a76-4fcc-8024-be92989c5863 None admin] RBAC: Adding query filter params (name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:46.142839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09d77a-0a76-4fcc-8024-be92989c5863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:46.144249 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09d77a-0a76-4fcc-8024-be92989c5863 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:46.152432 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 205/396] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Oct 12 14:26:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:46.153192 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09d77a-0a76-4fcc-8024-be92989c5863 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:46.188302 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f0a51-a08e-421c-a67f-a2c566230d59 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:46.221839 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e44fa0-3199-4ef7-9050-09921fd81191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:46.222493 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e44fa0-3199-4ef7-9050-09921fd81191 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:46.222774 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e44fa0-3199-4ef7-9050-09921fd81191 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:46.223588 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e44fa0-3199-4ef7-9050-09921fd81191 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:46.230125 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e44fa0-3199-4ef7-9050-09921fd81191 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:46.231611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e44fa0-3199-4ef7-9050-09921fd81191 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:46.242793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-d1e44fa0-3199-4ef7-9050-09921fd81191 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:46.242793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:46.242793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:46.242793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:46.242793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:46.242793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:46.242793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:46.242793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:46.242793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:46.242793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:46.242793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:46.242793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:46.242793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:26:46.242793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:26:46.242793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:46.243691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:46.244519 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:26:46.244519 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:26:46.244519 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:26:46.244519 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:26:46.244519 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:26:46.244519 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:26:46.244519 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:46.244519 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:46.244519 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 192/397] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:46.245101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e44fa0-3199-4ef7-9050-09921fd81191 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:46.253093 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:46.255850 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09d77a-0a76-4fcc-8024-be92989c5863 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:46.289674 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4138f7-9fec-4bea-9df1-674e4a87d29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:46.290298 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4138f7-9fec-4bea-9df1-674e4a87d29d None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:46.290657 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4138f7-9fec-4bea-9df1-674e4a87d29d None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:46.290922 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4138f7-9fec-4bea-9df1-674e4a87d29d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:46.298430 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4138f7-9fec-4bea-9df1-674e4a87d29d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:46.299888 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4138f7-9fec-4bea-9df1-674e4a87d29d None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:46.310969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-fd4138f7-9fec-4bea-9df1-674e4a87d29d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 14:26:46.310969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:46.310969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:46.310969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:46.310969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:46.310969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:46.310969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:46.310969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:46.310969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:46.310969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:46.310969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:46.310969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:46.310969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:26:46.310969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:26:46.310969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:26:46.310969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:46.311878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:46.312736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:46.312736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:46.312736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:26:46.312736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:26:46.312736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:26:46.312736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:26:46.312736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:26:46.312736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:26:46.312736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 14:26:46.312736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:46.312736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 206/398] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:46.313321 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4138f7-9fec-4bea-9df1-674e4a87d29d None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:46.315162 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:46.348968 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e44fa0-3199-4ef7-9050-09921fd81191 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:46.381637 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77d9ce-c8b8-4735-b183-a408bc35fd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:46.382237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77d9ce-c8b8-4735-b183-a408bc35fd22 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:46.382506 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77d9ce-c8b8-4735-b183-a408bc35fd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:46.382775 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77d9ce-c8b8-4735-b183-a408bc35fd22 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:46.383438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77d9ce-c8b8-4735-b183-a408bc35fd22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:46.384861 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77d9ce-c8b8-4735-b183-a408bc35fd22 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:46.405821 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 193/399] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:26:46 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:26:46.406614 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77d9ce-c8b8-4735-b183-a408bc35fd22 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:46.427104 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4138f7-9fec-4bea-9df1-674e4a87d29d None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:46.471861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3c412-ada0-4960-8050-e443ac6e55e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:46.472509 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3c412-ada0-4960-8050-e443ac6e55e1 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:46.472779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3c412-ada0-4960-8050-e443ac6e55e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:46.473141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3c412-ada0-4960-8050-e443ac6e55e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:46.473842 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3c412-ada0-4960-8050-e443ac6e55e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:46.474164 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3c412-ada0-4960-8050-e443ac6e55e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:46.475641 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3c412-ada0-4960-8050-e443ac6e55e1 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:46.481024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:46.497461 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 207/400] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:26:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:46.498169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3c412-ada0-4960-8050-e443ac6e55e1 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:46.533627 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77d9ce-c8b8-4735-b183-a408bc35fd22 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:46.565948 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:46.578566 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8272407-02ed-43cd-aff7-9ee6fa78d36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:46.579257 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8272407-02ed-43cd-aff7-9ee6fa78d36d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:46.579534 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8272407-02ed-43cd-aff7-9ee6fa78d36d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:46.579797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8272407-02ed-43cd-aff7-9ee6fa78d36d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:46.591857 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8272407-02ed-43cd-aff7-9ee6fa78d36d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:46.593331 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8272407-02ed-43cd-aff7-9ee6fa78d36d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:46.608810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-d8272407-02ed-43cd-aff7-9ee6fa78d36d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:46.608810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:46.608810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:46.608810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:46.608810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:46.608810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:46.608810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:46.608810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:46.608810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:46.608810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:46.608810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:46.608810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:46.608810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:46.608810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:46.608810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:46.608810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:46.609730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:46.610790 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:46.610790 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:46.610790 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:46.610790 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 194/401] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:46.610790 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8272407-02ed-43cd-aff7-9ee6fa78d36d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:46.621856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3c412-ada0-4960-8050-e443ac6e55e1 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:46.657587 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40615b7b-39e8-4fb7-8b89-a70a3b85f29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:46.658190 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40615b7b-39e8-4fb7-8b89-a70a3b85f29a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:46.658469 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40615b7b-39e8-4fb7-8b89-a70a3b85f29a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:46.658730 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40615b7b-39e8-4fb7-8b89-a70a3b85f29a None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:46.659447 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40615b7b-39e8-4fb7-8b89-a70a3b85f29a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:46.659753 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40615b7b-39e8-4fb7-8b89-a70a3b85f29a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:46.661751 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40615b7b-39e8-4fb7-8b89-a70a3b85f29a None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:46.674367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 208/402] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:26:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:46.675478 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40615b7b-39e8-4fb7-8b89-a70a3b85f29a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:46.716733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8272407-02ed-43cd-aff7-9ee6fa78d36d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:46.733083 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:46.755002 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1e19b-08e7-4eed-b8e3-f8386b94750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:46.755637 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1e19b-08e7-4eed-b8e3-f8386b94750c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:46.755914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1e19b-08e7-4eed-b8e3-f8386b94750c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:46.756187 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1e19b-08e7-4eed-b8e3-f8386b94750c None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:46.756919 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1e19b-08e7-4eed-b8e3-f8386b94750c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:46.757221 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1e19b-08e7-4eed-b8e3-f8386b94750c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:46.758645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1e19b-08e7-4eed-b8e3-f8386b94750c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:46.794253 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40615b7b-39e8-4fb7-8b89-a70a3b85f29a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:46.802106 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 195/403] 10.222.0.16 () {56 vars in 1137 bytes} [Wed Oct 12 14:26:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:46.802106 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1e19b-08e7-4eed-b8e3-f8386b94750c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:46.818976 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:46.831593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae0626-7016-45d6-9fc1-d98e5b8db693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:46.832841 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae0626-7016-45d6-9fc1-d98e5b8db693 None admin] REQUEST_METHOD: `PUT` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:46.833114 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae0626-7016-45d6-9fc1-d98e5b8db693 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:46.833388 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae0626-7016-45d6-9fc1-d98e5b8db693 None admin] PATH_INFO: `/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/2b95d3ba7df041aab3281129f6516d20/roles/745dcd2d05e447538a1aaff54b38193c` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:46.849389 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae0626-7016-45d6-9fc1-d98e5b8db693 None admin] RBAC: Authorizing `identity:create_grant(project_id=720b11237d2e48128c171be45fcd0ad0, user_id=2b95d3ba7df041aab3281129f6516d20, role_id=745dcd2d05e447538a1aaff54b38193c)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:46.850879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae0626-7016-45d6-9fc1-d98e5b8db693 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:46.875179 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 209/404] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:26:46 2022] PUT /identity/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/2b95d3ba7df041aab3281129f6516d20/roles/745dcd2d05e447538a1aaff54b38193c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:26:46.875179 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae0626-7016-45d6-9fc1-d98e5b8db693 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:46.945552 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1e19b-08e7-4eed-b8e3-f8386b94750c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:46.985068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9084a7-c45f-4231-970e-d795bc41d97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:46.985857 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9084a7-c45f-4231-970e-d795bc41d97a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:46.986231 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9084a7-c45f-4231-970e-d795bc41d97a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:46.986572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9084a7-c45f-4231-970e-d795bc41d97a None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:46.987342 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9084a7-c45f-4231-970e-d795bc41d97a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:46.987710 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9084a7-c45f-4231-970e-d795bc41d97a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:46.989810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9084a7-c45f-4231-970e-d795bc41d97a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:47.002972 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 196/405] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:47.004875 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9084a7-c45f-4231-970e-d795bc41d97a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:47.024971 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae0626-7016-45d6-9fc1-d98e5b8db693 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:47.041721 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:47.089519 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737970e-6ba5-4d74-bc00-d8721172a0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:47.090226 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737970e-6ba5-4d74-bc00-d8721172a0d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:47.090509 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737970e-6ba5-4d74-bc00-d8721172a0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:47.090843 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737970e-6ba5-4d74-bc00-d8721172a0d1 None admin] PATH_INFO: `/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/d0aeed5f32a1477dbd9d3e934669705b/roles/745dcd2d05e447538a1aaff54b38193c` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:47.120638 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a737970e-6ba5-4d74-bc00-d8721172a0d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=720b11237d2e48128c171be45fcd0ad0, user_id=d0aeed5f32a1477dbd9d3e934669705b, role_id=745dcd2d05e447538a1aaff54b38193c)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:47.122969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a737970e-6ba5-4d74-bc00-d8721172a0d1 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:47.133756 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9084a7-c45f-4231-970e-d795bc41d97a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:47.143765 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 210/406] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:26:46 2022] PUT /identity/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/d0aeed5f32a1477dbd9d3e934669705b/roles/745dcd2d05e447538a1aaff54b38193c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:26:47.144679 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737970e-6ba5-4d74-bc00-d8721172a0d1 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:47.194587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43d166-b9a6-4058-a60a-6701d248782f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:47.195575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43d166-b9a6-4058-a60a-6701d248782f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:47.195910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43d166-b9a6-4058-a60a-6701d248782f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:47.196187 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43d166-b9a6-4058-a60a-6701d248782f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:47.210783 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43d166-b9a6-4058-a60a-6701d248782f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:47.212499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43d166-b9a6-4058-a60a-6701d248782f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:47.231499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-8c43d166-b9a6-4058-a60a-6701d248782f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:47.231499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:47.231499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:47.231499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:47.231499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:47.231499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:47.231499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:47.231499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:47.231499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:47.231499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:47.231499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:47.231499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:47.231499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:47.231499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:47.231499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:47.231499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:47.232449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:47.233256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:47.233256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:47.233256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:47.233256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 197/407] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:47.240033 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43d166-b9a6-4058-a60a-6701d248782f None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:47.241253 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:47.341524 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:47.359958 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43d166-b9a6-4058-a60a-6701d248782f None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:47.380423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737970e-6ba5-4d74-bc00-d8721172a0d1 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:47.398223 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4339d39-9651-4392-a94d-108ca0ea4a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:47.399198 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4339d39-9651-4392-a94d-108ca0ea4a10 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:47.399614 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4339d39-9651-4392-a94d-108ca0ea4a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:47.400023 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4339d39-9651-4392-a94d-108ca0ea4a10 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:47.400806 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4339d39-9651-4392-a94d-108ca0ea4a10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:47.401246 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4339d39-9651-4392-a94d-108ca0ea4a10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:47.402789 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4339d39-9651-4392-a94d-108ca0ea4a10 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:47.417546 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 198/408] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:47.423180 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4339d39-9651-4392-a94d-108ca0ea4a10 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:47.426909 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ab56b-c608-4e46-9af1-732b8983fc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:47.427586 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ab56b-c608-4e46-9af1-732b8983fc3f None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:47.427856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ab56b-c608-4e46-9af1-732b8983fc3f None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:47.428179 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ab56b-c608-4e46-9af1-732b8983fc3f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:47.435927 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ab56b-c608-4e46-9af1-732b8983fc3f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:47.437595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ab56b-c608-4e46-9af1-732b8983fc3f None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:47.463425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-a06ab56b-c608-4e46-9af1-732b8983fc3f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 12 14:26:47.463425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:47.463425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:47.463425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:47.463425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:47.463425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:47.463425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:47.463425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:47.463425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:47.463425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:47.463425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:47.463425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:47.463425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:26:47.463425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:26:47.463425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:26:47.463425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:47.464575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:47.465636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:47.465636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:47.465636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:26:47.465636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:26:47.465636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:26:47.465636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:26:47.465636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:26:47.465636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:26:47.465636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 12 14:26:47.465636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:47.465636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 211/409] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Oct 12 14:26:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:47.473252 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ab56b-c608-4e46-9af1-732b8983fc3f None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:47.552002 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4339d39-9651-4392-a94d-108ca0ea4a10 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:47.595558 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de19d34-321c-4602-8a5b-b8e5c12b1f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:47.596418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de19d34-321c-4602-8a5b-b8e5c12b1f81 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:47.596839 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de19d34-321c-4602-8a5b-b8e5c12b1f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:47.597245 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de19d34-321c-4602-8a5b-b8e5c12b1f81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:47.604565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de19d34-321c-4602-8a5b-b8e5c12b1f81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:47.606182 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de19d34-321c-4602-8a5b-b8e5c12b1f81 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:47.618268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-1de19d34-321c-4602-8a5b-b8e5c12b1f81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:47.618268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:47.618268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:47.618268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:47.618268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:47.618268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:47.618268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:47.618268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:47.618268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:47.618268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:47.618268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:47.618268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:47.618268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:26:47.618268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:26:47.618268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:47.619313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:47.620338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:26:47.620338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:26:47.620338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:26:47.620338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:26:47.620338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:26:47.620338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:26:47.620338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:47.620338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:47.622229 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 199/410] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:47.627840 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de19d34-321c-4602-8a5b-b8e5c12b1f81 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:47.632297 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:47.659857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ab56b-c608-4e46-9af1-732b8983fc3f None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:47.670095 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:47.709823 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541d76-a1a0-4ba1-9e0a-0ee60f59b5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:47.710662 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541d76-a1a0-4ba1-9e0a-0ee60f59b5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:47.711008 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541d76-a1a0-4ba1-9e0a-0ee60f59b5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:47.711390 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541d76-a1a0-4ba1-9e0a-0ee60f59b5a0 None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:47.712121 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6541d76-a1a0-4ba1-9e0a-0ee60f59b5a0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:47.712423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6541d76-a1a0-4ba1-9e0a-0ee60f59b5a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:47.714005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6541d76-a1a0-4ba1-9e0a-0ee60f59b5a0 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:47.748817 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 212/411] 10.222.0.16 () {56 vars in 1139 bytes} [Wed Oct 12 14:26:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:47.754925 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541d76-a1a0-4ba1-9e0a-0ee60f59b5a0 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:47.762045 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de19d34-321c-4602-8a5b-b8e5c12b1f81 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:47.821232 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f710f6-a27b-4f14-830c-f69c280a8831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:47.821854 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f710f6-a27b-4f14-830c-f69c280a8831 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:47.822130 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f710f6-a27b-4f14-830c-f69c280a8831 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:47.822398 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f710f6-a27b-4f14-830c-f69c280a8831 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:47.823097 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f710f6-a27b-4f14-830c-f69c280a8831 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:47.823428 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f710f6-a27b-4f14-830c-f69c280a8831 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:47.824848 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f710f6-a27b-4f14-830c-f69c280a8831 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:47.862669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 200/412] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:26:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:47.866951 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541d76-a1a0-4ba1-9e0a-0ee60f59b5a0 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:47.868442 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f710f6-a27b-4f14-830c-f69c280a8831 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:47.903535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d03b0-86f7-4493-9a2c-7e19313db5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:47.904168 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d03b0-86f7-4493-9a2c-7e19313db5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:47.904441 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d03b0-86f7-4493-9a2c-7e19313db5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:47.904709 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d03b0-86f7-4493-9a2c-7e19313db5a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:47.919469 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:47.922038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d03b0-86f7-4493-9a2c-7e19313db5a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:47.923737 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d03b0-86f7-4493-9a2c-7e19313db5a1 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:47.951544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-761d03b0-86f7-4493-9a2c-7e19313db5a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:47.951544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:47.951544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:47.951544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:47.951544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:47.951544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:47.951544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:47.951544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:47.951544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:47.951544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:47.951544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:47.951544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:47.951544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:47.951544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:47.951544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:47.951544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:47.952484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:47.953289 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:47.953289 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:47.953289 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:47.959570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 213/413] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:47.963742 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:47.964140 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d03b0-86f7-4493-9a2c-7e19313db5a1 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:48.006956 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f710f6-a27b-4f14-830c-f69c280a8831 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:48.043803 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71749022-b98d-4a41-b4e7-974fd1517600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:48.044718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71749022-b98d-4a41-b4e7-974fd1517600 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:48.045133 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71749022-b98d-4a41-b4e7-974fd1517600 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:48.045540 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71749022-b98d-4a41-b4e7-974fd1517600 None admin] PATH_INFO: `/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/c8514f660bf949ada15e8d7fc6a062a2/roles/745dcd2d05e447538a1aaff54b38193c` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:48.060947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71749022-b98d-4a41-b4e7-974fd1517600 None admin] RBAC: Authorizing `identity:create_grant(project_id=720b11237d2e48128c171be45fcd0ad0, user_id=c8514f660bf949ada15e8d7fc6a062a2, role_id=745dcd2d05e447538a1aaff54b38193c)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:48.062599 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71749022-b98d-4a41-b4e7-974fd1517600 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:48.078835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 201/414] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:26:47 2022] PUT /identity/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/c8514f660bf949ada15e8d7fc6a062a2/roles/745dcd2d05e447538a1aaff54b38193c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 14:26:48.149887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d03b0-86f7-4493-9a2c-7e19313db5a1 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:48.194268 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c9c6b-7ebb-4e64-b15b-75b4eab3c65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:48.195041 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c9c6b-7ebb-4e64-b15b-75b4eab3c65d None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:48.195455 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c9c6b-7ebb-4e64-b15b-75b4eab3c65d None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:48.195763 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c9c6b-7ebb-4e64-b15b-75b4eab3c65d None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:48.196512 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c9c6b-7ebb-4e64-b15b-75b4eab3c65d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:48.196932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c9c6b-7ebb-4e64-b15b-75b4eab3c65d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:48.198610 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c9c6b-7ebb-4e64-b15b-75b4eab3c65d None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:48.220494 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 214/415] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:48.226546 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71749022-b98d-4a41-b4e7-974fd1517600 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:48.346502 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71749022-b98d-4a41-b4e7-974fd1517600 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:48.363511 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:48.387764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ab20f-a573-4622-b070-3ee84ffd60eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:48.388400 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ab20f-a573-4622-b070-3ee84ffd60eb None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:48.388670 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ab20f-a573-4622-b070-3ee84ffd60eb None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:48.388934 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ab20f-a573-4622-b070-3ee84ffd60eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:48.397353 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ab20f-a573-4622-b070-3ee84ffd60eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:48.398810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ab20f-a573-4622-b070-3ee84ffd60eb None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:48.412687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-cc4ab20f-a573-4622-b070-3ee84ffd60eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:48.412687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:48.412687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:48.412687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:48.412687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:48.412687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:48.412687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:48.412687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:48.412687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:48.412687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:48.412687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:48.412687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:48.412687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:26:48.412687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:26:48.412687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:48.413553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:48.414360 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:26:48.414360 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:26:48.414360 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:26:48.414360 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:26:48.414360 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:26:48.414360 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:26:48.414360 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:48.414360 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:48.414360 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 202/416] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:48.420477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c9c6b-7ebb-4e64-b15b-75b4eab3c65d None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:48.474562 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:48.525532 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c9c6b-7ebb-4e64-b15b-75b4eab3c65d None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:48.560851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fec38f-db2f-4a66-a932-bbe9c21ba430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:48.561456 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fec38f-db2f-4a66-a932-bbe9c21ba430 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:48.561727 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fec38f-db2f-4a66-a932-bbe9c21ba430 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:48.561989 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fec38f-db2f-4a66-a932-bbe9c21ba430 None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:48.562668 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fec38f-db2f-4a66-a932-bbe9c21ba430 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:48.562966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fec38f-db2f-4a66-a932-bbe9c21ba430 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:48.564438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fec38f-db2f-4a66-a932-bbe9c21ba430 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:48.577374 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 215/417] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:26:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:48.588235 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ab20f-a573-4622-b070-3ee84ffd60eb None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:48.709690 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ab20f-a573-4622-b070-3ee84ffd60eb None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:48.746840 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a584a2-c1e2-41a9-9cf3-6ac7b59a1faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:48.747531 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a584a2-c1e2-41a9-9cf3-6ac7b59a1faf None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:48.747810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a584a2-c1e2-41a9-9cf3-6ac7b59a1faf None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:48.748081 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a584a2-c1e2-41a9-9cf3-6ac7b59a1faf None admin] PATH_INFO: `/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/9d0d533e36ff4a5186f8704480b58646/roles/745dcd2d05e447538a1aaff54b38193c` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:48.763560 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a584a2-c1e2-41a9-9cf3-6ac7b59a1faf None admin] RBAC: Authorizing `identity:create_grant(project_id=720b11237d2e48128c171be45fcd0ad0, user_id=9d0d533e36ff4a5186f8704480b58646, role_id=745dcd2d05e447538a1aaff54b38193c)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:48.765068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a584a2-c1e2-41a9-9cf3-6ac7b59a1faf None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:48.783538 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 203/418] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:26:48 2022] PUT /identity/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/9d0d533e36ff4a5186f8704480b58646/roles/745dcd2d05e447538a1aaff54b38193c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:26:48.994873 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f69c1-cebd-4e07-be4f-76d2ccdd317e None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:48.995398 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f69c1-cebd-4e07-be4f-76d2ccdd317e None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:48.995398 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f69c1-cebd-4e07-be4f-76d2ccdd317e None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:48.996920 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 216/419] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:49.002884 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb30d96-8156-4398-a48d-92f6e911a7de None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:49.003190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb30d96-8156-4398-a48d-92f6e911a7de None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:49.003456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb30d96-8156-4398-a48d-92f6e911a7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:49.023135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3cb30d96-8156-4398-a48d-92f6e911a7de None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:49.024340 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3cb30d96-8156-4398-a48d-92f6e911a7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:49.044164 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:49.101637 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 204/420] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:49.375504 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f69c1-cebd-4e07-be4f-76d2ccdd317e None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:49.494337 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f69c1-cebd-4e07-be4f-76d2ccdd317e None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:49.515506 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:49.540761 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb38b0-095e-4e2c-980e-504a434c8613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:49.541394 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb38b0-095e-4e2c-980e-504a434c8613 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:49.541668 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb38b0-095e-4e2c-980e-504a434c8613 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:49.541931 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb38b0-095e-4e2c-980e-504a434c8613 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:49.556689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb38b0-095e-4e2c-980e-504a434c8613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:49.561656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-5ccb38b0-095e-4e2c-980e-504a434c8613 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:49.561656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:49.561656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:49.561656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:49.561656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:49.561656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:49.561656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:49.561656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:49.561656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:49.561656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:49.561656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:49.561656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:49.561656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:49.561656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:49.561656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:49.561656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:49.563754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:49.563754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:49.563754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:49.563754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:49.563754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:49.563754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:49.563754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:49.563754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:49.563754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:49.563754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:49.563754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:49.563754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:49.563754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:49.563754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:49.563754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:49.563754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:49.565673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:49.565673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:49.565673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:49.565673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:49.565673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:49.565673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:49.565673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:49.565673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:49.565673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:49.565673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:49.565673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:49.565673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:49.565673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 217/421] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Oct 12 14:26:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:49.572401 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb30d96-8156-4398-a48d-92f6e911a7de None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:49.592207 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11390e84-c494-4961-9c26-dd41a70ccdac None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:49.592481 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11390e84-c494-4961-9c26-dd41a70ccdac None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:49.592745 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11390e84-c494-4961-9c26-dd41a70ccdac None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:49.594300 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 218/422] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:49.612472 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf36fcc-7449-4694-a436-b5476deb9dea None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:49.612753 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf36fcc-7449-4694-a436-b5476deb9dea None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:49.613640 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf36fcc-7449-4694-a436-b5476deb9dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:49.645687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-bcf36fcc-7449-4694-a436-b5476deb9dea None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:49.647009 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-bcf36fcc-7449-4694-a436-b5476deb9dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:49.713750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb30d96-8156-4398-a48d-92f6e911a7de None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:49.749411 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760947be-3d61-4a72-924b-24feef146e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:49.750021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760947be-3d61-4a72-924b-24feef146e0e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:49.750291 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760947be-3d61-4a72-924b-24feef146e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:49.750556 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760947be-3d61-4a72-924b-24feef146e0e None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:49.751233 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760947be-3d61-4a72-924b-24feef146e0e None admin] RBAC: Adding query filter params (name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:49.751534 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760947be-3d61-4a72-924b-24feef146e0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:49.752889 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760947be-3d61-4a72-924b-24feef146e0e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:49.762352 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 205/423] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Oct 12 14:26:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:49.767803 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760947be-3d61-4a72-924b-24feef146e0e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:49.790035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:49.844265 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 219/424] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:49.878296 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760947be-3d61-4a72-924b-24feef146e0e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:49.929721 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa42b5-1e78-4c1b-be64-7f6e49aa5696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:49.930361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa42b5-1e78-4c1b-be64-7f6e49aa5696 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:49.930631 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa42b5-1e78-4c1b-be64-7f6e49aa5696 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:49.930896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa42b5-1e78-4c1b-be64-7f6e49aa5696 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:49.944579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e308883-e659-4156-b274-34311ecf5eb4 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:49.944886 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e308883-e659-4156-b274-34311ecf5eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:49.945124 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e308883-e659-4156-b274-34311ecf5eb4 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:49.947502 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 220/425] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:26:49.949669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa42b5-1e78-4c1b-be64-7f6e49aa5696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:49.951136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa42b5-1e78-4c1b-be64-7f6e49aa5696 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:49.952890 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211142d2-f4c5-44f1-b72d-7ce0f9c2b99c None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:49.953170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211142d2-f4c5-44f1-b72d-7ce0f9c2b99c None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:49.953445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211142d2-f4c5-44f1-b72d-7ce0f9c2b99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:49.971261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-e8aa42b5-1e78-4c1b-be64-7f6e49aa5696 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:49.971261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:49.971261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:49.971261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:49.971261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:49.971261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:49.971261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:49.971261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:49.971261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:49.971261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:49.971261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:49.971261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:49.971261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:49.971261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:49.971261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:49.971261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:49.972237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:49.973564 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:49.973564 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:49.973564 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:49.973564 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 206/426] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:49.975244 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-211142d2-f4c5-44f1-b72d-7ce0f9c2b99c None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:49.976424 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-211142d2-f4c5-44f1-b72d-7ce0f9c2b99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:49.978915 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa42b5-1e78-4c1b-be64-7f6e49aa5696 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:50.108042 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 221/427] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:26:50.108943 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:50.124270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211142d2-f4c5-44f1-b72d-7ce0f9c2b99c None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:50.150961 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa42b5-1e78-4c1b-be64-7f6e49aa5696 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:50.173642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:50.215495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182af3bd-92a0-424e-8b76-155a8d112572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:50.216409 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182af3bd-92a0-424e-8b76-155a8d112572 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:50.216892 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182af3bd-92a0-424e-8b76-155a8d112572 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:50.217366 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182af3bd-92a0-424e-8b76-155a8d112572 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:50.218187 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182af3bd-92a0-424e-8b76-155a8d112572 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:50.218674 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182af3bd-92a0-424e-8b76-155a8d112572 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:50.221020 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182af3bd-92a0-424e-8b76-155a8d112572 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:50.247220 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 207/428] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:50.249538 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182af3bd-92a0-424e-8b76-155a8d112572 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:50.309896 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211142d2-f4c5-44f1-b72d-7ce0f9c2b99c None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:50.345273 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422cf589-9bb5-400a-8897-590614b61216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:50.345892 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cf589-9bb5-400a-8897-590614b61216 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:50.346159 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cf589-9bb5-400a-8897-590614b61216 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:50.346424 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cf589-9bb5-400a-8897-590614b61216 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:50.364731 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422cf589-9bb5-400a-8897-590614b61216 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:50.369832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-422cf589-9bb5-400a-8897-590614b61216 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:50.369832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:50.369832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:50.369832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:50.369832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:50.369832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:50.369832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:50.369832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:50.369832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:50.369832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:50.369832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:50.369832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:50.369832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:50.369832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:50.369832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:50.369832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:50.370686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:50.370686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:50.370686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:50.370686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:50.370686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:50.370686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:50.370686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:50.370686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:50.370686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:50.370686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:50.370686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:50.370686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:50.370686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:50.370686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:50.370686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:50.370686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:50.371497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:50.371497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:50.371497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:50.371497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:50.371497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:50.371497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:50.371497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:50.371497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:50.371497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:50.371497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:50.371497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:50.371497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:50.372124 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 222/429] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Oct 12 14:26:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:50.379221 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422cf589-9bb5-400a-8897-590614b61216 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:50.403961 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182af3bd-92a0-424e-8b76-155a8d112572 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:50.461442 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:50.469207 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c36570-22e3-4741-a6da-61cf20da97c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:50.470121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c36570-22e3-4741-a6da-61cf20da97c2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:50.470605 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c36570-22e3-4741-a6da-61cf20da97c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:50.471059 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c36570-22e3-4741-a6da-61cf20da97c2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:50.479452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c36570-22e3-4741-a6da-61cf20da97c2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:50.481371 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c36570-22e3-4741-a6da-61cf20da97c2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:50.496740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-44c36570-22e3-4741-a6da-61cf20da97c2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 12 14:26:50.496740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:50.496740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:50.496740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:50.496740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:50.496740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:50.496740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:50.496740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:50.496740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:50.496740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:50.496740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:50.496740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:50.496740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:26:50.496740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:26:50.496740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:26:50.496740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:50.497657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:50.498522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:50.498522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:50.498522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:26:50.498522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:26:50.498522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:26:50.498522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:26:50.498522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:26:50.498522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:26:50.498522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 12 14:26:50.498522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:50.499008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 208/430] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:26:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:50.499566 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c36570-22e3-4741-a6da-61cf20da97c2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:50.521361 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:50.534113 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422cf589-9bb5-400a-8897-590614b61216 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:50.579049 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cc3da-2bd7-4cd9-88fb-28fd25a887f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:50.579941 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cc3da-2bd7-4cd9-88fb-28fd25a887f8 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:50.580354 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cc3da-2bd7-4cd9-88fb-28fd25a887f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:50.580779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cc3da-2bd7-4cd9-88fb-28fd25a887f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:50.582154 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552cc3da-2bd7-4cd9-88fb-28fd25a887f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:50.582743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552cc3da-2bd7-4cd9-88fb-28fd25a887f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:50.584771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552cc3da-2bd7-4cd9-88fb-28fd25a887f8 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:50.602473 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 223/431] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Oct 12 14:26:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:50.602473 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cc3da-2bd7-4cd9-88fb-28fd25a887f8 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:50.627601 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c36570-22e3-4741-a6da-61cf20da97c2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:50.669440 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6013b3-09e1-4857-a69c-58f0e9471b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:50.670714 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6013b3-09e1-4857-a69c-58f0e9471b9a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:50.671209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6013b3-09e1-4857-a69c-58f0e9471b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:50.671699 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6013b3-09e1-4857-a69c-58f0e9471b9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:50.683457 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6013b3-09e1-4857-a69c-58f0e9471b9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:50.689737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-6e6013b3-09e1-4857-a69c-58f0e9471b9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:50.689737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:50.689737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:50.689737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:50.689737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:50.689737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:50.689737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:50.689737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:50.689737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:50.689737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:50.689737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:50.689737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:50.689737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:50.689737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:50.689737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:50.689737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:50.690596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:50.690596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:50.690596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:50.690596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:50.690596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:50.690596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:50.690596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:50.690596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:50.690596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:50.690596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:50.690596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:50.690596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:50.690596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:50.690596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:50.690596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:50.690596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:50.692330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:50.692330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:50.692330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:50.692330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:50.692330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:50.692330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:50.692330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:50.692330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:50.692330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:50.692330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:50.692330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:50.692330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:50.694668 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 209/432] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Oct 12 14:26:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:50.695680 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6013b3-09e1-4857-a69c-58f0e9471b9a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:50.725036 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cc3da-2bd7-4cd9-88fb-28fd25a887f8 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:50.750151 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:50.773512 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434a320-5601-403f-9b2e-2b629f096ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:50.774292 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434a320-5601-403f-9b2e-2b629f096ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:50.774699 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434a320-5601-403f-9b2e-2b629f096ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:50.774988 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434a320-5601-403f-9b2e-2b629f096ed1 None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:50.775853 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434a320-5601-403f-9b2e-2b629f096ed1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:50.776157 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434a320-5601-403f-9b2e-2b629f096ed1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:50.778732 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434a320-5601-403f-9b2e-2b629f096ed1 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:50.807550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 224/433] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Oct 12 14:26:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:50.807550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434a320-5601-403f-9b2e-2b629f096ed1 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:50.817022 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6013b3-09e1-4857-a69c-58f0e9471b9a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:50.824929 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:50.858036 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636aec6-f1d2-4b3c-8d0d-e855ee3f4c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:50.859177 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636aec6-f1d2-4b3c-8d0d-e855ee3f4c0e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:50.859674 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636aec6-f1d2-4b3c-8d0d-e855ee3f4c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:50.860130 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636aec6-f1d2-4b3c-8d0d-e855ee3f4c0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:50.871657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636aec6-f1d2-4b3c-8d0d-e855ee3f4c0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:50.873427 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636aec6-f1d2-4b3c-8d0d-e855ee3f4c0e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:50.889094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-0636aec6-f1d2-4b3c-8d0d-e855ee3f4c0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:50.889094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:50.889094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:50.889094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:50.889094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:50.889094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:50.889094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:50.889094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:50.889094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:50.889094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:50.889094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:50.889094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:50.889094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:50.889094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:50.889094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:50.889094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:50.890072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:50.890905 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:50.890905 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:50.890905 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:50.898753 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 210/434] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:50.900161 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636aec6-f1d2-4b3c-8d0d-e855ee3f4c0e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:50.926285 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434a320-5601-403f-9b2e-2b629f096ed1 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:50.962597 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5e86d-42d8-4f96-b250-91ab58b23a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:50.963233 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5e86d-42d8-4f96-b250-91ab58b23a61 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:50.963508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5e86d-42d8-4f96-b250-91ab58b23a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:50.963777 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5e86d-42d8-4f96-b250-91ab58b23a61 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:50.964442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5e86d-42d8-4f96-b250-91ab58b23a61 None admin] RBAC: Adding query filter params (name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:50.964985 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5e86d-42d8-4f96-b250-91ab58b23a61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:50.966085 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5e86d-42d8-4f96-b250-91ab58b23a61 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:50.975525 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 225/435] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Oct 12 14:26:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:50.976339 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5e86d-42d8-4f96-b250-91ab58b23a61 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:51.011407 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:51.015953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636aec6-f1d2-4b3c-8d0d-e855ee3f4c0e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:51.054652 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f558a8-2d4a-4c05-93be-b3534ee4279e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:51.055652 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f558a8-2d4a-4c05-93be-b3534ee4279e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:51.056123 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f558a8-2d4a-4c05-93be-b3534ee4279e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:51.056580 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f558a8-2d4a-4c05-93be-b3534ee4279e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:51.058955 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:51.071209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f558a8-2d4a-4c05-93be-b3534ee4279e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:51.072938 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f558a8-2d4a-4c05-93be-b3534ee4279e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:51.087855 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5e86d-42d8-4f96-b250-91ab58b23a61 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:51.088469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-e4f558a8-2d4a-4c05-93be-b3534ee4279e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:51.088469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:51.088469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:51.088469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:51.088469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:51.088469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:51.088469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:51.088469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:51.088469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:51.088469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:51.088469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:51.088469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:51.088469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:51.088469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:51.088469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:51.088469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:51.089338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:51.090156 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:51.090156 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:51.090156 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:51.090156 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 211/436] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:51.090371 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f558a8-2d4a-4c05-93be-b3534ee4279e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:51.134798 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb1f39-6aaf-41b2-9002-af65945019c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:51.135526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb1f39-6aaf-41b2-9002-af65945019c8 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:51.135713 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb1f39-6aaf-41b2-9002-af65945019c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:51.136087 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb1f39-6aaf-41b2-9002-af65945019c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:51.136719 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb1f39-6aaf-41b2-9002-af65945019c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:51.137015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb1f39-6aaf-41b2-9002-af65945019c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:51.138501 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb1f39-6aaf-41b2-9002-af65945019c8 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:51.155646 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 226/437] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:51.155646 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3397811-f80a-4ea2-bbf8-dd8d0d2377e7 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:51.155646 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3397811-f80a-4ea2-bbf8-dd8d0d2377e7 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:51.155646 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3397811-f80a-4ea2-bbf8-dd8d0d2377e7 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:51.155646 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 227/438] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:51.155646 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3397811-f80a-4ea2-bbf8-dd8d0d2377e7 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:51.201905 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f558a8-2d4a-4c05-93be-b3534ee4279e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:51.238252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575f50e-2eca-4f4e-ab07-dff9154ef0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:51.238816 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575f50e-2eca-4f4e-ab07-dff9154ef0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:51.239086 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575f50e-2eca-4f4e-ab07-dff9154ef0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:51.239490 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575f50e-2eca-4f4e-ab07-dff9154ef0d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:51.250582 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575f50e-2eca-4f4e-ab07-dff9154ef0d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:51.252267 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575f50e-2eca-4f4e-ab07-dff9154ef0d4 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:51.258788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3397811-f80a-4ea2-bbf8-dd8d0d2377e7 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:51.268087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-b575f50e-2eca-4f4e-ab07-dff9154ef0d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:51.268087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:51.268087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:51.268087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:51.268087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:51.268087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:51.268087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:51.268087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:51.268087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:51.268087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:51.268087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:51.268087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:51.268087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:51.268087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:51.268087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:51.268087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:51.269080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:51.270072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:51.270072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:51.270072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:51.270072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 212/439] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:51.270837 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575f50e-2eca-4f4e-ab07-dff9154ef0d4 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:51.274383 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:51.298193 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8f2be-c171-4539-a88d-6becac8d35d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:51.298805 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8f2be-c171-4539-a88d-6becac8d35d0 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:51.299130 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8f2be-c171-4539-a88d-6becac8d35d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:51.299433 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8f2be-c171-4539-a88d-6becac8d35d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:51.300056 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b8f2be-c171-4539-a88d-6becac8d35d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:51.301099 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b8f2be-c171-4539-a88d-6becac8d35d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:51.302457 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b8f2be-c171-4539-a88d-6becac8d35d0 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:51.304054 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:51.320404 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 228/440] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:51.320404 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac596b1f-1ed2-44ca-8125-acd635b6f570 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:51.320404 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac596b1f-1ed2-44ca-8125-acd635b6f570 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:51.320404 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac596b1f-1ed2-44ca-8125-acd635b6f570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:51.339567 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-ac596b1f-1ed2-44ca-8125-acd635b6f570 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:51.340718 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-ac596b1f-1ed2-44ca-8125-acd635b6f570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:51.392629 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575f50e-2eca-4f4e-ab07-dff9154ef0d4 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:51.415464 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 229/441] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:51.416216 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac596b1f-1ed2-44ca-8125-acd635b6f570 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:51.441390 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb443fd-93f2-47d1-af26-c597032b867c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:51.442644 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb443fd-93f2-47d1-af26-c597032b867c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:51.443119 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb443fd-93f2-47d1-af26-c597032b867c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:51.443740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb443fd-93f2-47d1-af26-c597032b867c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:51.454253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb443fd-93f2-47d1-af26-c597032b867c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:51.456698 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb443fd-93f2-47d1-af26-c597032b867c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:51.471671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-2eb443fd-93f2-47d1-af26-c597032b867c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 12 14:26:51.471671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:51.471671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:51.471671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:51.471671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:51.471671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:51.471671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:51.471671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:51.471671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:51.471671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:51.471671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:51.471671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:51.471671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:26:51.471671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:26:51.471671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:26:51.471671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:51.473136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:51.473980 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:51.473980 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:51.473980 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:26:51.473980 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:26:51.473980 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:26:51.473980 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:26:51.473980 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:26:51.473980 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:26:51.473980 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 12 14:26:51.473980 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:51.476321 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 213/442] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:51.477295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb443fd-93f2-47d1-af26-c597032b867c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:51.533485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac596b1f-1ed2-44ca-8125-acd635b6f570 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:51.543892 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:51.567333 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:51.583729 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb443fd-93f2-47d1-af26-c597032b867c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:51.584222 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b117b-c555-43ed-be2e-92938f1be4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:51.584836 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b117b-c555-43ed-be2e-92938f1be4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:51.585307 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b117b-c555-43ed-be2e-92938f1be4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:51.585584 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b117b-c555-43ed-be2e-92938f1be4a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:51.587061 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b117b-c555-43ed-be2e-92938f1be4a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:51.587395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b117b-c555-43ed-be2e-92938f1be4a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:51.588784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b117b-c555-43ed-be2e-92938f1be4a5 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:51.606173 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 230/443] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:51.606173 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b117b-c555-43ed-be2e-92938f1be4a5 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:51.630808 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c008e-93b9-480b-b979-378978699e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:51.631471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c008e-93b9-480b-b979-378978699e08 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:51.631751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c008e-93b9-480b-b979-378978699e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:51.632023 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c008e-93b9-480b-b979-378978699e08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:51.638760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c008e-93b9-480b-b979-378978699e08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:51.640233 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c008e-93b9-480b-b979-378978699e08 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:51.651873 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-9d1c008e-93b9-480b-b979-378978699e08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:51.651873 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:51.651873 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:51.651873 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:51.651873 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:51.651873 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:51.651873 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:51.651873 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:51.651873 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:51.651873 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:51.651873 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:51.651873 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:51.651873 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:26:51.651873 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:26:51.651873 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:51.652728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:51.653544 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:26:51.653544 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:26:51.653544 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:26:51.653544 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:26:51.653544 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:26:51.653544 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:26:51.653544 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:51.653544 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:51.653544 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 214/444] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:51.654851 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765010f0-1117-4ba5-aee4-313dff4ba174 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:51.654851 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765010f0-1117-4ba5-aee4-313dff4ba174 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:51.654851 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765010f0-1117-4ba5-aee4-313dff4ba174 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:51.659261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 215/445] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:51.659261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765010f0-1117-4ba5-aee4-313dff4ba174 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:51.715657 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b117b-c555-43ed-be2e-92938f1be4a5 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:51.749239 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f58874-1086-4160-b3cf-0c1c9cef9153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:51.749842 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f58874-1086-4160-b3cf-0c1c9cef9153 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:51.750116 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f58874-1086-4160-b3cf-0c1c9cef9153 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:51.750388 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f58874-1086-4160-b3cf-0c1c9cef9153 None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:51.751093 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f58874-1086-4160-b3cf-0c1c9cef9153 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:51.751426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f58874-1086-4160-b3cf-0c1c9cef9153 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:51.752866 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f58874-1086-4160-b3cf-0c1c9cef9153 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:51.760249 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765010f0-1117-4ba5-aee4-313dff4ba174 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:51.769999 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 231/446] 10.222.0.16 () {56 vars in 1137 bytes} [Wed Oct 12 14:26:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:51.770795 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f58874-1086-4160-b3cf-0c1c9cef9153 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:51.786584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:51.797492 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747a6af-497b-43e2-b5ee-748d149a9729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:51.798127 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747a6af-497b-43e2-b5ee-748d149a9729 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:51.798402 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747a6af-497b-43e2-b5ee-748d149a9729 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:51.798667 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747a6af-497b-43e2-b5ee-748d149a9729 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:51.806377 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747a6af-497b-43e2-b5ee-748d149a9729 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:51.807833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747a6af-497b-43e2-b5ee-748d149a9729 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:51.815056 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:51.819105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-8747a6af-497b-43e2-b5ee-748d149a9729 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 14:26:51.819105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:51.819105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:51.819105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:51.819105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:51.819105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:51.819105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:51.819105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:51.819105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:51.819105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:51.819105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:51.819105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:51.819105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:26:51.819105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:26:51.819105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:26:51.819105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:51.820317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:51.821174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:51.821174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:51.821174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:26:51.821174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:26:51.821174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:26:51.821174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:26:51.821174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:26:51.821174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:26:51.821174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 14:26:51.821174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:51.821174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 216/447] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Oct 12 14:26:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:51.821719 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839b642-815f-4712-a8ea-3fed9e6add16 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:51.821719 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839b642-815f-4712-a8ea-3fed9e6add16 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:51.822022 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839b642-815f-4712-a8ea-3fed9e6add16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:51.839539 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-8839b642-815f-4712-a8ea-3fed9e6add16 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:51.840724 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8839b642-815f-4712-a8ea-3fed9e6add16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:51.874092 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f58874-1086-4160-b3cf-0c1c9cef9153 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:51.906617 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 217/448] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:51.907054 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839b642-815f-4712-a8ea-3fed9e6add16 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:51.907455 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72244a95-7fa4-4271-913f-253ba5be03a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:51.908059 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72244a95-7fa4-4271-913f-253ba5be03a6 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:51.908328 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72244a95-7fa4-4271-913f-253ba5be03a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:51.908589 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72244a95-7fa4-4271-913f-253ba5be03a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:51.909261 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72244a95-7fa4-4271-913f-253ba5be03a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:51.909840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72244a95-7fa4-4271-913f-253ba5be03a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:51.911295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72244a95-7fa4-4271-913f-253ba5be03a6 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:51.931367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 232/449] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:26:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:51.932352 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72244a95-7fa4-4271-913f-253ba5be03a6 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:52.014067 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839b642-815f-4712-a8ea-3fed9e6add16 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:52.028929 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:52.037535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72244a95-7fa4-4271-913f-253ba5be03a6 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:52.039116 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:52.052907 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0e6ca-b87d-4b3c-9b12-02a3833de58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:52.053530 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0e6ca-b87d-4b3c-9b12-02a3833de58e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:52.053801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0e6ca-b87d-4b3c-9b12-02a3833de58e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:52.054064 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0e6ca-b87d-4b3c-9b12-02a3833de58e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:52.064168 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0e6ca-b87d-4b3c-9b12-02a3833de58e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:52.069461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-84e0e6ca-b87d-4b3c-9b12-02a3833de58e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:52.069461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:52.069461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:52.069461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:52.069461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:52.069461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:52.069461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:52.069461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:52.069461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:52.069461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:52.069461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:52.069461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:52.069461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:52.069461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:52.069461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:52.069461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:52.070406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:52.070406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:52.070406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:52.070406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:52.070406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:52.070406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:52.070406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:52.070406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:52.070406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:52.070406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:52.070406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:52.070406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:52.070406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:52.070406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:52.070406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:52.070406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:52.071180 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:52.071180 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:52.071180 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:52.071180 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:52.071180 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:52.071180 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:52.071180 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:52.071180 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:52.071180 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:52.071180 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:52.071180 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:52.071180 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:52.071180 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 218/450] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Oct 12 14:26:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:52.071960 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0e6ca-b87d-4b3c-9b12-02a3833de58e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:52.080809 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb480e90-787e-4128-b40c-6086a7a65e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:52.081953 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb480e90-787e-4128-b40c-6086a7a65e79 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:52.082228 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb480e90-787e-4128-b40c-6086a7a65e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:52.082492 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb480e90-787e-4128-b40c-6086a7a65e79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:52.152466 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb480e90-787e-4128-b40c-6086a7a65e79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:52.153913 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb480e90-787e-4128-b40c-6086a7a65e79 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:52.188118 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0e6ca-b87d-4b3c-9b12-02a3833de58e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:52.223106 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96bdfd-84f4-4f4c-ba3c-fba489f54eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:52.223730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96bdfd-84f4-4f4c-ba3c-fba489f54eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:52.224005 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96bdfd-84f4-4f4c-ba3c-fba489f54eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:52.224272 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96bdfd-84f4-4f4c-ba3c-fba489f54eb1 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:52.224991 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96bdfd-84f4-4f4c-ba3c-fba489f54eb1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:52.225290 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96bdfd-84f4-4f4c-ba3c-fba489f54eb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:52.226706 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96bdfd-84f4-4f4c-ba3c-fba489f54eb1 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:52.243163 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 219/451] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Oct 12 14:26:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:52.243942 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96bdfd-84f4-4f4c-ba3c-fba489f54eb1 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:52.255267 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 233/452] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 14:26:52.256090 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb480e90-787e-4128-b40c-6086a7a65e79 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:52.258249 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:52.278732 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:52.346215 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96bdfd-84f4-4f4c-ba3c-fba489f54eb1 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:52.354549 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb480e90-787e-4128-b40c-6086a7a65e79 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:52.379037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b2eca-2cef-4c8f-bf53-8a49915f87da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:52.379717 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b2eca-2cef-4c8f-bf53-8a49915f87da None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:52.379995 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b2eca-2cef-4c8f-bf53-8a49915f87da None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:52.380265 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b2eca-2cef-4c8f-bf53-8a49915f87da None admin] PATH_INFO: `/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/c8381a7c961047c69de894cf5b0055b6/roles/745dcd2d05e447538a1aaff54b38193c` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:52.388488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d48fc-74af-4621-8627-23ade17e68c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:52.389082 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d48fc-74af-4621-8627-23ade17e68c3 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:52.389355 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d48fc-74af-4621-8627-23ade17e68c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:52.389621 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d48fc-74af-4621-8627-23ade17e68c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:52.390275 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d48fc-74af-4621-8627-23ade17e68c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:52.390577 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d48fc-74af-4621-8627-23ade17e68c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:52.391961 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d48fc-74af-4621-8627-23ade17e68c3 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:52.395256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b2eca-2cef-4c8f-bf53-8a49915f87da None admin] RBAC: Authorizing `identity:create_grant(project_id=720b11237d2e48128c171be45fcd0ad0, user_id=c8381a7c961047c69de894cf5b0055b6, role_id=745dcd2d05e447538a1aaff54b38193c)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:52.397012 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b2eca-2cef-4c8f-bf53-8a49915f87da None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:52.400204 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 234/453] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Oct 12 14:26:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:52.400991 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d48fc-74af-4621-8627-23ade17e68c3 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:52.410275 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 220/454] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:26:52 2022] PUT /identity/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/c8381a7c961047c69de894cf5b0055b6/roles/745dcd2d05e447538a1aaff54b38193c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:26:52.416497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b2eca-2cef-4c8f-bf53-8a49915f87da None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:52.478889 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:52.514970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:52.522459 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d48fc-74af-4621-8627-23ade17e68c3 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:52.537504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b2eca-2cef-4c8f-bf53-8a49915f87da None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:52.558599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194beac-f6fc-455e-841d-6cd2fb363495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:52.559275 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194beac-f6fc-455e-841d-6cd2fb363495 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:52.559553 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194beac-f6fc-455e-841d-6cd2fb363495 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:52.559819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194beac-f6fc-455e-841d-6cd2fb363495 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:52.571039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194beac-f6fc-455e-841d-6cd2fb363495 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:52.576539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-7194beac-f6fc-455e-841d-6cd2fb363495 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:52.576539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:52.576539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:52.576539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:52.576539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:52.576539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:52.576539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:52.576539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:52.576539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:52.576539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:52.576539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:52.576539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:52.576539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:52.576539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:52.576539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:52.576539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:52.577672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:52.577672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:52.577672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:52.577672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:52.577672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:52.577672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:52.577672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:52.577672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:52.577672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:52.577672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:52.577672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:52.577672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:52.577672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:52.577672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:52.577672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:52.577672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:52.578649 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:52.578649 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:52.578649 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:52.578649 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:52.578649 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:52.578649 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:52.578649 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:52.578649 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:52.578649 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:52.578649 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:52.578649 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:52.578649 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:52.581674 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8aacf-6695-484e-ab4f-f8b717dcc7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:52.582339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8aacf-6695-484e-ab4f-f8b717dcc7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:52.582617 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8aacf-6695-484e-ab4f-f8b717dcc7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:52.583100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8aacf-6695-484e-ab4f-f8b717dcc7d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:52.583243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 235/455] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Oct 12 14:26:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:52.584018 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194beac-f6fc-455e-841d-6cd2fb363495 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:52.657426 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8aacf-6695-484e-ab4f-f8b717dcc7d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:52.659306 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8aacf-6695-484e-ab4f-f8b717dcc7d2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:52.692252 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194beac-f6fc-455e-841d-6cd2fb363495 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:52.717203 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:52.729345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be088a-dca4-4e16-b45e-be5689006338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:52.729974 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be088a-dca4-4e16-b45e-be5689006338 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:52.730252 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be088a-dca4-4e16-b45e-be5689006338 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:52.730517 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be088a-dca4-4e16-b45e-be5689006338 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:52.742876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be088a-dca4-4e16-b45e-be5689006338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:52.744354 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be088a-dca4-4e16-b45e-be5689006338 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:52.751718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:52.760908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-b1be088a-dca4-4e16-b45e-be5689006338 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:52.760908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:52.760908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:52.760908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:52.760908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:52.760908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:52.760908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:52.760908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:52.760908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:52.760908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:52.760908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:52.760908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:52.760908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:52.760908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:52.760908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:52.760908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:52.761814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:52.762606 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:52.762606 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:52.762606 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:52.766743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 236/456] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:52.767603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be088a-dca4-4e16-b45e-be5689006338 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:52.782998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 221/457] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 14:26:52.782998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8aacf-6695-484e-ab4f-f8b717dcc7d2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:52.876354 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be088a-dca4-4e16-b45e-be5689006338 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:52.891190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8aacf-6695-484e-ab4f-f8b717dcc7d2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:52.916943 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b86a3a-3d9f-4a76-bcbe-52ae9b64c4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:52.917764 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b86a3a-3d9f-4a76-bcbe-52ae9b64c4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:52.918176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b86a3a-3d9f-4a76-bcbe-52ae9b64c4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:52.918443 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b86a3a-3d9f-4a76-bcbe-52ae9b64c4d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:52.929590 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b86a3a-3d9f-4a76-bcbe-52ae9b64c4d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:52.930988 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d4afc-c2f3-41c4-b2b1-6d6d2a179306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:52.931202 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b86a3a-3d9f-4a76-bcbe-52ae9b64c4d7 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:52.931651 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d4afc-c2f3-41c4-b2b1-6d6d2a179306 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:52.931929 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d4afc-c2f3-41c4-b2b1-6d6d2a179306 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:52.932196 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d4afc-c2f3-41c4-b2b1-6d6d2a179306 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:52.933023 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d4afc-c2f3-41c4-b2b1-6d6d2a179306 None admin] RBAC: Adding query filter params (name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:52.933335 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d4afc-c2f3-41c4-b2b1-6d6d2a179306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:52.935017 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d4afc-c2f3-41c4-b2b1-6d6d2a179306 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:52.948126 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 222/458] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Oct 12 14:26:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:52.948126 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d4afc-c2f3-41c4-b2b1-6d6d2a179306 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:52.948338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-19b86a3a-3d9f-4a76-bcbe-52ae9b64c4d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:52.948338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:52.948338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:52.948338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:52.948338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:52.948338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:52.948338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:52.948338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:52.948338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:52.948338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:52.948338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:52.948338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:52.948338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:52.948338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:52.948338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:52.948338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:52.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:52.950056 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:52.950056 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:52.950056 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:52.950056 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 237/459] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:52.950056 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b86a3a-3d9f-4a76-bcbe-52ae9b64c4d7 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:52.981427 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:53.001088 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:53.057817 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d4afc-c2f3-41c4-b2b1-6d6d2a179306 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:53.068236 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b86a3a-3d9f-4a76-bcbe-52ae9b64c4d7 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:53.092557 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f205c-0eae-4853-b3c7-c025edc05771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:53.093189 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f205c-0eae-4853-b3c7-c025edc05771 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:53.093458 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f205c-0eae-4853-b3c7-c025edc05771 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:53.093720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f205c-0eae-4853-b3c7-c025edc05771 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:53.094354 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f205c-0eae-4853-b3c7-c025edc05771 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:53.094650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f205c-0eae-4853-b3c7-c025edc05771 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:53.096052 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f205c-0eae-4853-b3c7-c025edc05771 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:53.107111 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56329a3a-e1de-4e9d-9dd5-3190d7856872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:53.107769 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56329a3a-e1de-4e9d-9dd5-3190d7856872 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:53.108052 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56329a3a-e1de-4e9d-9dd5-3190d7856872 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:53.113354 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56329a3a-e1de-4e9d-9dd5-3190d7856872 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:53.113482 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 223/460] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:53.113482 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f205c-0eae-4853-b3c7-c025edc05771 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:53.124696 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56329a3a-e1de-4e9d-9dd5-3190d7856872 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:53.126162 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56329a3a-e1de-4e9d-9dd5-3190d7856872 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:53.141932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-56329a3a-e1de-4e9d-9dd5-3190d7856872 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:53.141932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:53.141932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:53.141932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:53.141932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:53.141932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:53.141932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:53.141932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:53.141932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:53.141932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:53.141932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:53.141932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:53.141932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:53.141932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:53.141932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:53.141932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:53.142857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:53.144314 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:53.144314 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:53.144314 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:53.148487 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 238/461] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:53.149325 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56329a3a-e1de-4e9d-9dd5-3190d7856872 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:53.220806 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f205c-0eae-4853-b3c7-c025edc05771 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:53.224276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:53.234349 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:53.254788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56329a3a-e1de-4e9d-9dd5-3190d7856872 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:53.259454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3540d4-f4d0-45bb-9db6-962fea8c19b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:53.260074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3540d4-f4d0-45bb-9db6-962fea8c19b9 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:53.260345 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3540d4-f4d0-45bb-9db6-962fea8c19b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:53.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3540d4-f4d0-45bb-9db6-962fea8c19b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:53.271333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3540d4-f4d0-45bb-9db6-962fea8c19b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:53.273154 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3540d4-f4d0-45bb-9db6-962fea8c19b9 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:53.290604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-cb3540d4-f4d0-45bb-9db6-962fea8c19b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:53.290604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:53.290604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:53.290604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:53.290604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:53.290604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:53.290604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:53.290604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:53.290604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:53.290604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:53.290604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:53.290604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:53.290604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:53.290604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:53.290604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:53.290604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:53.291788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:53.293025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:53.293025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:53.293025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:53.293188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df2a47-0f34-4e30-972e-fb06bfb7ad28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:53.293273 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 224/462] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:53.293273 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3540d4-f4d0-45bb-9db6-962fea8c19b9 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:53.296630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df2a47-0f34-4e30-972e-fb06bfb7ad28 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:53.297688 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df2a47-0f34-4e30-972e-fb06bfb7ad28 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:53.297688 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df2a47-0f34-4e30-972e-fb06bfb7ad28 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:53.297905 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df2a47-0f34-4e30-972e-fb06bfb7ad28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:53.298193 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df2a47-0f34-4e30-972e-fb06bfb7ad28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:53.299620 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df2a47-0f34-4e30-972e-fb06bfb7ad28 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:53.317068 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 239/463] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:53.317068 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df2a47-0f34-4e30-972e-fb06bfb7ad28 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:53.406982 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3540d4-f4d0-45bb-9db6-962fea8c19b9 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:53.418350 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df2a47-0f34-4e30-972e-fb06bfb7ad28 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:53.444535 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47028bc-b604-4b22-8949-6ed72efeaccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:53.445267 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47028bc-b604-4b22-8949-6ed72efeaccd None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:53.445540 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47028bc-b604-4b22-8949-6ed72efeaccd None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:53.445804 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47028bc-b604-4b22-8949-6ed72efeaccd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:53.452821 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af3710-6401-4a4c-b9ec-6617c8608ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:53.453430 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47028bc-b604-4b22-8949-6ed72efeaccd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:53.453611 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af3710-6401-4a4c-b9ec-6617c8608ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:53.453868 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af3710-6401-4a4c-b9ec-6617c8608ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:53.454133 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af3710-6401-4a4c-b9ec-6617c8608ce8 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:53.454754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af3710-6401-4a4c-b9ec-6617c8608ce8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:53.454845 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47028bc-b604-4b22-8949-6ed72efeaccd None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:53.455047 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af3710-6401-4a4c-b9ec-6617c8608ce8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:53.457319 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af3710-6401-4a4c-b9ec-6617c8608ce8 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:53.465442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:53.466791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-c47028bc-b604-4b22-8949-6ed72efeaccd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:53.466791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:53.466791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:53.466791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:53.466791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:53.466791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:53.466791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:53.466791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:53.466791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:53.466791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:53.466791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:53.466791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:53.466791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:26:53.466791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:26:53.466791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:53.467826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:53.474183 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:26:53.474183 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:26:53.474183 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:26:53.474183 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:26:53.474183 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:26:53.474183 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:26:53.474183 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:53.474183 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:53.474607 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 240/464] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:53.474607 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af3710-6401-4a4c-b9ec-6617c8608ce8 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:53.474729 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 225/465] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:53.474729 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47028bc-b604-4b22-8949-6ed72efeaccd None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:53.494096 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:53.587513 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47028bc-b604-4b22-8949-6ed72efeaccd None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:53.591021 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af3710-6401-4a4c-b9ec-6617c8608ce8 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:53.622795 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391408ab-4d57-4d9e-8768-76249fba39f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:53.623452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391408ab-4d57-4d9e-8768-76249fba39f6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:53.623724 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391408ab-4d57-4d9e-8768-76249fba39f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:53.624141 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391408ab-4d57-4d9e-8768-76249fba39f6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:53.625523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e62fe-525f-4faf-97de-f7813c074db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:53.626125 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e62fe-525f-4faf-97de-f7813c074db4 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:53.626397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e62fe-525f-4faf-97de-f7813c074db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:53.626664 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e62fe-525f-4faf-97de-f7813c074db4 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:53.627327 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e62fe-525f-4faf-97de-f7813c074db4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:53.627633 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e62fe-525f-4faf-97de-f7813c074db4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:53.629535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e62fe-525f-4faf-97de-f7813c074db4 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:53.633384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391408ab-4d57-4d9e-8768-76249fba39f6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:53.634985 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391408ab-4d57-4d9e-8768-76249fba39f6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:53.641633 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 241/466] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:53.642449 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e62fe-525f-4faf-97de-f7813c074db4 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:53.647763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-391408ab-4d57-4d9e-8768-76249fba39f6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 14:26:53.647763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:53.647763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:53.647763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:53.647763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:53.647763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:53.647763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:53.647763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:53.647763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:53.647763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:53.647763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:53.647763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:53.647763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:26:53.647763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:26:53.647763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:26:53.647763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:53.648809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:53.649864 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:53.649864 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:53.649864 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:26:53.649864 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:26:53.649864 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:26:53.649864 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:26:53.649864 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:26:53.649864 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:26:53.649864 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 14:26:53.649864 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:53.653035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 226/467] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:53.653965 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391408ab-4d57-4d9e-8768-76249fba39f6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:53.709084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:53.745715 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:53.750905 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e62fe-525f-4faf-97de-f7813c074db4 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:53.772687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391408ab-4d57-4d9e-8768-76249fba39f6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:53.788971 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45635565-a142-4b32-bd6f-6e4e0af40095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:53.789657 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45635565-a142-4b32-bd6f-6e4e0af40095 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:53.789849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45635565-a142-4b32-bd6f-6e4e0af40095 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:53.790089 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45635565-a142-4b32-bd6f-6e4e0af40095 None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:53.790761 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45635565-a142-4b32-bd6f-6e4e0af40095 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:53.791182 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45635565-a142-4b32-bd6f-6e4e0af40095 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:53.792508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45635565-a142-4b32-bd6f-6e4e0af40095 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:53.808676 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 242/468] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:26:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:53.808676 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45635565-a142-4b32-bd6f-6e4e0af40095 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:53.814037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c316a1-93af-4879-8038-71744346340c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:53.814673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c316a1-93af-4879-8038-71744346340c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:53.815232 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c316a1-93af-4879-8038-71744346340c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:53.816037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c316a1-93af-4879-8038-71744346340c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:53.822734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c316a1-93af-4879-8038-71744346340c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:53.824219 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c316a1-93af-4879-8038-71744346340c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:53.836474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-37c316a1-93af-4879-8038-71744346340c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:53.836474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:53.836474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:53.836474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:53.836474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:53.836474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:53.836474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:53.836474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:53.836474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:53.836474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:53.836474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:53.836474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:53.836474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:26:53.836474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:26:53.836474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:53.837286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:53.838132 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:26:53.838132 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:26:53.838132 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:26:53.838132 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:26:53.838132 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:26:53.838132 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:26:53.838132 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:53.838132 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:53.838132 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 227/469] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:53.839224 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c316a1-93af-4879-8038-71744346340c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:53.918492 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45635565-a142-4b32-bd6f-6e4e0af40095 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:53.938735 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:53.946243 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c316a1-93af-4879-8038-71744346340c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:53.954372 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a5c10-da23-47c9-9e71-bff9f53c67a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:53.955135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a5c10-da23-47c9-9e71-bff9f53c67a5 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:53.955533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a5c10-da23-47c9-9e71-bff9f53c67a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:53.955935 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a5c10-da23-47c9-9e71-bff9f53c67a5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:53.964534 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a5c10-da23-47c9-9e71-bff9f53c67a5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:53.966003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a5c10-da23-47c9-9e71-bff9f53c67a5 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:53.977722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-867a5c10-da23-47c9-9e71-bff9f53c67a5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 12 14:26:53.977722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:53.977722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:53.977722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:53.977722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:53.977722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:53.977722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:53.977722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:53.977722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:53.977722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:53.977722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:53.977722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:53.977722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:26:53.977722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:26:53.977722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:26:53.977722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:53.978757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:53.984498 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:53.984498 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:53.984498 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:26:53.984498 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:26:53.984498 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:26:53.984498 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:26:53.984498 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:26:53.984498 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:26:53.984498 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 12 14:26:53.984498 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:53.984498 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 243/470] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Oct 12 14:26:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:53.984498 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a5c10-da23-47c9-9e71-bff9f53c67a5 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:53.988035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3aec47-05db-4752-9a02-704685bfc79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:53.988945 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3aec47-05db-4752-9a02-704685bfc79e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:53.989227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3aec47-05db-4752-9a02-704685bfc79e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:53.989542 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3aec47-05db-4752-9a02-704685bfc79e None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:53.990302 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3aec47-05db-4752-9a02-704685bfc79e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:53.990649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3aec47-05db-4752-9a02-704685bfc79e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:53.992314 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3aec47-05db-4752-9a02-704685bfc79e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:53.996383 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:54.014778 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 228/471] 10.222.0.16 () {56 vars in 1137 bytes} [Wed Oct 12 14:26:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:54.014778 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3aec47-05db-4752-9a02-704685bfc79e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:54.087370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a5c10-da23-47c9-9e71-bff9f53c67a5 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:54.121280 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1f13b-e5da-47cd-8566-2b9549cf0f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:54.121994 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1f13b-e5da-47cd-8566-2b9549cf0f8f None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:54.122336 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1f13b-e5da-47cd-8566-2b9549cf0f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:54.122662 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1f13b-e5da-47cd-8566-2b9549cf0f8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:54.123488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1f13b-e5da-47cd-8566-2b9549cf0f8f None admin] RBAC: Adding query filter params (role.id=745dcd2d05e447538a1aaff54b38193c, scope.project.id=720b11237d2e48128c171be45fcd0ad0, user.id=2b95d3ba7df041aab3281129f6516d20) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:54.123850 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1f13b-e5da-47cd-8566-2b9549cf0f8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:54.125355 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1f13b-e5da-47cd-8566-2b9549cf0f8f None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:54.127049 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3aec47-05db-4752-9a02-704685bfc79e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:54.139183 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 244/472] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:26:53 2022] GET /identity/v3/role_assignments?user.id=2b95d3ba7df041aab3281129f6516d20&scope.project.id=720b11237d2e48128c171be45fcd0ad0&role.id=745dcd2d05e447538a1aaff54b38193c => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:54.139183 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1f13b-e5da-47cd-8566-2b9549cf0f8f None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:54.193380 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7e1ae-d199-498c-8088-9fe91b80fc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:54.194146 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7e1ae-d199-498c-8088-9fe91b80fc09 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:54.194489 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7e1ae-d199-498c-8088-9fe91b80fc09 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:54.195040 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7e1ae-d199-498c-8088-9fe91b80fc09 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:54.195910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7e1ae-d199-498c-8088-9fe91b80fc09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:54.196448 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7e1ae-d199-498c-8088-9fe91b80fc09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:54.198057 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7e1ae-d199-498c-8088-9fe91b80fc09 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:54.207255 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:54.212758 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 229/473] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:26:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:54.218591 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7e1ae-d199-498c-8088-9fe91b80fc09 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:54.280980 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1f13b-e5da-47cd-8566-2b9549cf0f8f None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:54.294165 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:54.323032 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499b372-9782-44d7-8e1e-86561c2afa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:54.323903 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499b372-9782-44d7-8e1e-86561c2afa70 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:54.324315 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499b372-9782-44d7-8e1e-86561c2afa70 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:54.324720 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499b372-9782-44d7-8e1e-86561c2afa70 None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:54.325648 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499b372-9782-44d7-8e1e-86561c2afa70 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:54.326306 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499b372-9782-44d7-8e1e-86561c2afa70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:54.327534 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499b372-9782-44d7-8e1e-86561c2afa70 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:54.346274 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7e1ae-d199-498c-8088-9fe91b80fc09 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:54.348025 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 245/474] 10.222.0.16 () {56 vars in 1139 bytes} [Wed Oct 12 14:26:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:54.348663 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499b372-9782-44d7-8e1e-86561c2afa70 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:54.391807 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9d36f-a576-4648-948c-12502c9763d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:54.392775 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9d36f-a576-4648-948c-12502c9763d8 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:54.394557 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9d36f-a576-4648-948c-12502c9763d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:54.395060 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9d36f-a576-4648-948c-12502c9763d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:54.476927 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499b372-9782-44d7-8e1e-86561c2afa70 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:54.478812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:54.493252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9d36f-a576-4648-948c-12502c9763d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:54.494882 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9d36f-a576-4648-948c-12502c9763d8 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:54.516234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe1dd3-9bb9-43ed-bca7-ebd0539b0065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:54.516888 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe1dd3-9bb9-43ed-bca7-ebd0539b0065 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:54.517168 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe1dd3-9bb9-43ed-bca7-ebd0539b0065 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:54.517440 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe1dd3-9bb9-43ed-bca7-ebd0539b0065 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:54.518163 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe1dd3-9bb9-43ed-bca7-ebd0539b0065 None admin] RBAC: Adding query filter params (role.id=745dcd2d05e447538a1aaff54b38193c, scope.project.id=720b11237d2e48128c171be45fcd0ad0, user.id=d0aeed5f32a1477dbd9d3e934669705b) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:54.518614 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe1dd3-9bb9-43ed-bca7-ebd0539b0065 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:54.520079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe1dd3-9bb9-43ed-bca7-ebd0539b0065 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:54.526817 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 246/475] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:26:54 2022] GET /identity/v3/role_assignments?user.id=d0aeed5f32a1477dbd9d3e934669705b&scope.project.id=720b11237d2e48128c171be45fcd0ad0&role.id=745dcd2d05e447538a1aaff54b38193c => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:54.535480 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe1dd3-9bb9-43ed-bca7-ebd0539b0065 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:54.559080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:54.619203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 230/476] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 14:26:54.621089 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0ee27-1d54-4c67-b866-18168ac067b9 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:54.621528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0ee27-1d54-4c67-b866-18168ac067b9 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:54.621940 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0ee27-1d54-4c67-b866-18168ac067b9 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:54.628175 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 231/477] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:54.628175 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0ee27-1d54-4c67-b866-18168ac067b9 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:54.681638 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe1dd3-9bb9-43ed-bca7-ebd0539b0065 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:54.722602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d1cfb-e468-405b-969e-b384041664fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:54.723575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d1cfb-e468-405b-969e-b384041664fb None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:54.723992 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d1cfb-e468-405b-969e-b384041664fb None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:54.724512 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d1cfb-e468-405b-969e-b384041664fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:54.751907 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0ee27-1d54-4c67-b866-18168ac067b9 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:54.762295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:54.797270 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad54fb-54d2-4afd-abd9-b5a35a6410cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:54.797270 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad54fb-54d2-4afd-abd9-b5a35a6410cc None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:54.797270 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad54fb-54d2-4afd-abd9-b5a35a6410cc None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:54.797270 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad54fb-54d2-4afd-abd9-b5a35a6410cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:54.807368 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d1cfb-e468-405b-969e-b384041664fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:54.808821 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad54fb-54d2-4afd-abd9-b5a35a6410cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:54.808937 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d1cfb-e468-405b-969e-b384041664fb None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:54.810401 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad54fb-54d2-4afd-abd9-b5a35a6410cc None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:54.827019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-a4ad54fb-54d2-4afd-abd9-b5a35a6410cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:54.827019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:54.827019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:54.827019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:54.827019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:54.827019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:54.827019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:54.827019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:54.827019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:54.827019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:54.827019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:54.827019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:54.827019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:54.827019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:54.827019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:54.827019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:54.828120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:54.829137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:54.829137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:54.829137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:54.829137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 232/478] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:54.829729 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b81da2-fb7c-44a1-a2ce-c30ce8f3a8f7 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:54.830004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b81da2-fb7c-44a1-a2ce-c30ce8f3a8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:54.830269 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b81da2-fb7c-44a1-a2ce-c30ce8f3a8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:54.845764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:54.853340 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-22b81da2-fb7c-44a1-a2ce-c30ce8f3a8f7 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:54.854957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-22b81da2-fb7c-44a1-a2ce-c30ce8f3a8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:54.923134 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 247/479] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 14:26:54.928580 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d1cfb-e468-405b-969e-b384041664fb None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:54.947315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 233/480] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:54.948585 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b81da2-fb7c-44a1-a2ce-c30ce8f3a8f7 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:55.037340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:55.099215 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d1cfb-e468-405b-969e-b384041664fb None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:55.124415 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b81da2-fb7c-44a1-a2ce-c30ce8f3a8f7 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:55.145981 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2fc0d-0d61-4017-914f-758d7d547716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:55.146739 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2fc0d-0d61-4017-914f-758d7d547716 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:55.147014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2fc0d-0d61-4017-914f-758d7d547716 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:55.147308 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2fc0d-0d61-4017-914f-758d7d547716 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:55.147941 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2fc0d-0d61-4017-914f-758d7d547716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:55.148232 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2fc0d-0d61-4017-914f-758d7d547716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:55.149594 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2fc0d-0d61-4017-914f-758d7d547716 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:55.162118 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 248/481] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:55.166999 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2fc0d-0d61-4017-914f-758d7d547716 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:55.193136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80beeffd-d880-492a-b93d-85d8cde3a144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:55.193763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80beeffd-d880-492a-b93d-85d8cde3a144 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:55.194037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80beeffd-d880-492a-b93d-85d8cde3a144 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:55.194298 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80beeffd-d880-492a-b93d-85d8cde3a144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:55.208402 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:55.215171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80beeffd-d880-492a-b93d-85d8cde3a144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:55.216634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80beeffd-d880-492a-b93d-85d8cde3a144 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:55.248495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-80beeffd-d880-492a-b93d-85d8cde3a144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:55.248495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:55.248495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:55.248495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:55.248495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:55.248495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:55.248495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:55.248495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:55.248495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:55.248495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:55.248495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:55.248495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:55.248495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:55.248495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:55.248495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:55.248495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:55.249575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:55.250375 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:55.250375 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:55.250375 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:55.251946 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 234/482] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:55.255869 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80beeffd-d880-492a-b93d-85d8cde3a144 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:55.344651 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2fc0d-0d61-4017-914f-758d7d547716 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:55.379750 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:55.393417 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69aa70-bd2e-42a8-81c9-4ee54ba441bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:55.394429 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69aa70-bd2e-42a8-81c9-4ee54ba441bf None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:55.394922 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69aa70-bd2e-42a8-81c9-4ee54ba441bf None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:55.395100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80beeffd-d880-492a-b93d-85d8cde3a144 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:55.395843 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69aa70-bd2e-42a8-81c9-4ee54ba441bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:55.405228 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69aa70-bd2e-42a8-81c9-4ee54ba441bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:55.407638 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69aa70-bd2e-42a8-81c9-4ee54ba441bf None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:55.425370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-cb69aa70-bd2e-42a8-81c9-4ee54ba441bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:55.425370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:55.425370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:55.425370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:55.425370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:55.425370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:55.425370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:55.425370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:55.425370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:55.425370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:55.425370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:55.425370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:55.425370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:26:55.425370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:26:55.425370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:55.426210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:55.427103 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:26:55.427103 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:26:55.427103 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:26:55.427103 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:26:55.427103 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:26:55.427103 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:26:55.427103 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:55.427103 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:55.433647 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 249/483] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:55.433647 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69aa70-bd2e-42a8-81c9-4ee54ba441bf None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:55.439162 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782341e0-6015-4d18-856c-a5b1a316be25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:55.440601 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782341e0-6015-4d18-856c-a5b1a316be25 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:55.441077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782341e0-6015-4d18-856c-a5b1a316be25 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:55.441553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782341e0-6015-4d18-856c-a5b1a316be25 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:55.443022 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782341e0-6015-4d18-856c-a5b1a316be25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:55.443551 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782341e0-6015-4d18-856c-a5b1a316be25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:55.446078 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782341e0-6015-4d18-856c-a5b1a316be25 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:55.463916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 235/484] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:55.465791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782341e0-6015-4d18-856c-a5b1a316be25 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:55.521195 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:55.563120 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69aa70-bd2e-42a8-81c9-4ee54ba441bf None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:55.590606 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782341e0-6015-4d18-856c-a5b1a316be25 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:55.603344 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154d3ad-efd0-4a8a-985d-713085bda136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:55.604626 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154d3ad-efd0-4a8a-985d-713085bda136 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:55.605358 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154d3ad-efd0-4a8a-985d-713085bda136 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:55.605828 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154d3ad-efd0-4a8a-985d-713085bda136 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:55.619970 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f154d3ad-efd0-4a8a-985d-713085bda136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:55.627301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-f154d3ad-efd0-4a8a-985d-713085bda136 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:55.627301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:55.627301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:55.627301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:55.627301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:55.627301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:55.627301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:55.627301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:55.627301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:55.627301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:55.627301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:55.627301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:55.627301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:55.627301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:55.627301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:55.627301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:55.628170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:55.628170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:55.628170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:55.628170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:55.628170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:55.628170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:55.628170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:55.628170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:55.628170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:55.628170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:55.628170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:55.628170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:55.628170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:55.628170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:55.628170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:55.628170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:55.629014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:55.629014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:55.629014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:55.629014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:55.629014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:55.629014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:55.629014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:55.629014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:55.629014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:55.629014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:55.629014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:26:55.629014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:55.629604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a1e3f-68ab-4d8b-82be-77ce2d2a9da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:55.629604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a1e3f-68ab-4d8b-82be-77ce2d2a9da6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:55.629604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a1e3f-68ab-4d8b-82be-77ce2d2a9da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:55.629604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a1e3f-68ab-4d8b-82be-77ce2d2a9da6 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:55.629920 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a1e3f-68ab-4d8b-82be-77ce2d2a9da6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:55.630514 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a1e3f-68ab-4d8b-82be-77ce2d2a9da6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:55.631505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a1e3f-68ab-4d8b-82be-77ce2d2a9da6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:55.637116 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 250/485] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Oct 12 14:26:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:55.637116 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154d3ad-efd0-4a8a-985d-713085bda136 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:55.651187 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 236/486] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:26:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:55.651187 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a1e3f-68ab-4d8b-82be-77ce2d2a9da6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:55.695910 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:55.763905 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154d3ad-efd0-4a8a-985d-713085bda136 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:55.769018 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a1e3f-68ab-4d8b-82be-77ce2d2a9da6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:55.778647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:55.807034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b2bbb-13f3-4c2b-b61f-b8f98d1ae4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:55.807253 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d05ebc-523b-4f5f-91ee-7f8864f7b39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:55.807968 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d05ebc-523b-4f5f-91ee-7f8864f7b39a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:55.808172 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b2bbb-13f3-4c2b-b61f-b8f98d1ae4be None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:55.808446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d05ebc-523b-4f5f-91ee-7f8864f7b39a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:55.808745 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b2bbb-13f3-4c2b-b61f-b8f98d1ae4be None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:55.808869 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d05ebc-523b-4f5f-91ee-7f8864f7b39a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:55.809269 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b2bbb-13f3-4c2b-b61f-b8f98d1ae4be None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:55.810063 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b2bbb-13f3-4c2b-b61f-b8f98d1ae4be None admin] RBAC: Adding query filter params (name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:55.810501 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b2bbb-13f3-4c2b-b61f-b8f98d1ae4be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:55.812053 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b2bbb-13f3-4c2b-b61f-b8f98d1ae4be None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:55.816716 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d05ebc-523b-4f5f-91ee-7f8864f7b39a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:55.818782 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d05ebc-523b-4f5f-91ee-7f8864f7b39a None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:55.821796 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 237/487] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Oct 12 14:26:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:55.822899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b2bbb-13f3-4c2b-b61f-b8f98d1ae4be None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:55.837486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-e7d05ebc-523b-4f5f-91ee-7f8864f7b39a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:55.837486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:55.837486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:55.837486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:55.837486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:55.837486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:55.837486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:55.837486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:55.837486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:55.837486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:55.837486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:55.837486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:55.837486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:26:55.837486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:26:55.837486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:55.838418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:55.839255 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:26:55.839255 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:26:55.839255 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:26:55.839255 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:26:55.839255 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:26:55.839255 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:26:55.839255 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:55.839255 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:55.839255 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 251/488] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:55.840449 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d05ebc-523b-4f5f-91ee-7f8864f7b39a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:55.936337 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b2bbb-13f3-4c2b-b61f-b8f98d1ae4be None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:55.962224 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d05ebc-523b-4f5f-91ee-7f8864f7b39a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:55.969406 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:55.974403 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15c7d1-c321-41ce-a5c6-99aa0791963d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:55.975646 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15c7d1-c321-41ce-a5c6-99aa0791963d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:55.976176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15c7d1-c321-41ce-a5c6-99aa0791963d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:55.976626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15c7d1-c321-41ce-a5c6-99aa0791963d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:55.977523 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15c7d1-c321-41ce-a5c6-99aa0791963d None admin] RBAC: Adding query filter params (role.id=745dcd2d05e447538a1aaff54b38193c, scope.project.id=720b11237d2e48128c171be45fcd0ad0, user.id=c8514f660bf949ada15e8d7fc6a062a2) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:55.978001 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15c7d1-c321-41ce-a5c6-99aa0791963d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:55.979645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15c7d1-c321-41ce-a5c6-99aa0791963d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:55.986369 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 238/489] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:26:55 2022] GET /identity/v3/role_assignments?user.id=c8514f660bf949ada15e8d7fc6a062a2&scope.project.id=720b11237d2e48128c171be45fcd0ad0&role.id=745dcd2d05e447538a1aaff54b38193c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:55.987467 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15c7d1-c321-41ce-a5c6-99aa0791963d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:56.020709 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e2a39-5938-4921-ba63-57c2d7ede0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:56.021467 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e2a39-5938-4921-ba63-57c2d7ede0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:56.021808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e2a39-5938-4921-ba63-57c2d7ede0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:56.022160 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e2a39-5938-4921-ba63-57c2d7ede0e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:56.032235 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:56.038420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e2a39-5938-4921-ba63-57c2d7ede0e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:56.040011 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e2a39-5938-4921-ba63-57c2d7ede0e2 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:56.064323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-fd3e2a39-5938-4921-ba63-57c2d7ede0e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:56.064323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:56.064323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:56.064323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:56.064323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:56.064323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:56.064323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:56.064323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:56.064323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:56.064323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:56.064323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:56.064323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:56.064323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:56.064323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:56.064323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:56.064323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:56.065851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:56.066654 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:56.066654 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:56.066654 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:56.075769 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 252/490] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:56.076550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e2a39-5938-4921-ba63-57c2d7ede0e2 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:56.116068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15c7d1-c321-41ce-a5c6-99aa0791963d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:56.215992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2aad6-a354-4197-9a73-7a13b6a49822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:56.216547 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2aad6-a354-4197-9a73-7a13b6a49822 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:56.216823 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2aad6-a354-4197-9a73-7a13b6a49822 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:56.217086 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2aad6-a354-4197-9a73-7a13b6a49822 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:56.217788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2aad6-a354-4197-9a73-7a13b6a49822 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:56.218084 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2aad6-a354-4197-9a73-7a13b6a49822 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:56.219562 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2aad6-a354-4197-9a73-7a13b6a49822 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:56.270733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 239/491] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:26:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:56.280564 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2aad6-a354-4197-9a73-7a13b6a49822 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:56.341068 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e2a39-5938-4921-ba63-57c2d7ede0e2 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:56.415429 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95f6e2-3f32-477b-a1fe-e8cdd4142712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:56.416535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95f6e2-3f32-477b-a1fe-e8cdd4142712 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:56.416535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95f6e2-3f32-477b-a1fe-e8cdd4142712 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:56.416535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95f6e2-3f32-477b-a1fe-e8cdd4142712 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:56.417206 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95f6e2-3f32-477b-a1fe-e8cdd4142712 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:56.417459 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95f6e2-3f32-477b-a1fe-e8cdd4142712 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:56.425438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95f6e2-3f32-477b-a1fe-e8cdd4142712 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:56.456350 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 253/492] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:56.464674 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:56.465774 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95f6e2-3f32-477b-a1fe-e8cdd4142712 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:56.475395 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2aad6-a354-4197-9a73-7a13b6a49822 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:56.493508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:56.526886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1326ebb-81ee-4ece-9987-f9d70e1d8eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:56.527907 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1326ebb-81ee-4ece-9987-f9d70e1d8eba None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:56.528403 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1326ebb-81ee-4ece-9987-f9d70e1d8eba None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:56.529017 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1326ebb-81ee-4ece-9987-f9d70e1d8eba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:56.529951 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1326ebb-81ee-4ece-9987-f9d70e1d8eba None admin] RBAC: Adding query filter params (role.id=745dcd2d05e447538a1aaff54b38193c, scope.project.id=720b11237d2e48128c171be45fcd0ad0, user.id=9d0d533e36ff4a5186f8704480b58646) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:56.530536 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1326ebb-81ee-4ece-9987-f9d70e1d8eba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:56.537787 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1326ebb-81ee-4ece-9987-f9d70e1d8eba None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:56.552999 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 240/493] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:26:56 2022] GET /identity/v3/role_assignments?user.id=9d0d533e36ff4a5186f8704480b58646&scope.project.id=720b11237d2e48128c171be45fcd0ad0&role.id=745dcd2d05e447538a1aaff54b38193c => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:56.632717 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a66d2-edd9-47d5-b7c7-635d870f2064 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:56.633044 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a66d2-edd9-47d5-b7c7-635d870f2064 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:56.633272 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a66d2-edd9-47d5-b7c7-635d870f2064 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:56.635037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 241/494] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:56.641061 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad73fc1-6972-4dce-ac61-20278c31c9ee None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:56.641347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad73fc1-6972-4dce-ac61-20278c31c9ee None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:56.642104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad73fc1-6972-4dce-ac61-20278c31c9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:56.648483 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95f6e2-3f32-477b-a1fe-e8cdd4142712 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:56.664234 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9ad73fc1-6972-4dce-ac61-20278c31c9ee None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:56.665489 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9ad73fc1-6972-4dce-ac61-20278c31c9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:56.743740 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97b077-1858-4637-902e-d4ef84b04502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:56.744305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97b077-1858-4637-902e-d4ef84b04502 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:56.744581 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97b077-1858-4637-902e-d4ef84b04502 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:56.744846 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97b077-1858-4637-902e-d4ef84b04502 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:56.762741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97b077-1858-4637-902e-d4ef84b04502 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:56.764269 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97b077-1858-4637-902e-d4ef84b04502 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:56.776872 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 242/495] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:26:56.793851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-ff97b077-1858-4637-902e-d4ef84b04502 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 12 14:26:56.793851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:56.793851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:56.793851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:56.793851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:56.793851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:56.793851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:56.793851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:56.793851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:56.793851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:56.793851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:56.793851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:56.793851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:26:56.793851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:26:56.793851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:26:56.793851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:56.794822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:56.795808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:56.795808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:56.795808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:26:56.795808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:26:56.795808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:26:56.795808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:26:56.795808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:26:56.795808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:26:56.795808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 12 14:26:56.795808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:56.795808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 254/496] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:26:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:56.803192 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad73fc1-6972-4dce-ac61-20278c31c9ee None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:56.913327 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:56.926999 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad73fc1-6972-4dce-ac61-20278c31c9ee None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:56.967251 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc2a49-5cfb-4e08-a7c3-88c8830a5921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:56.967875 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc2a49-5cfb-4e08-a7c3-88c8830a5921 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:56.968146 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc2a49-5cfb-4e08-a7c3-88c8830a5921 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:56.968460 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc2a49-5cfb-4e08-a7c3-88c8830a5921 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:56.969201 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc2a49-5cfb-4e08-a7c3-88c8830a5921 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:56.969500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc2a49-5cfb-4e08-a7c3-88c8830a5921 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:56.970940 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc2a49-5cfb-4e08-a7c3-88c8830a5921 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:56.992692 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 243/497] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Oct 12 14:26:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:56.997939 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97b077-1858-4637-902e-d4ef84b04502 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:57.067753 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc2a49-5cfb-4e08-a7c3-88c8830a5921 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:57.097290 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:57.125956 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97b077-1858-4637-902e-d4ef84b04502 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:57.162317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc150d-4004-4508-a7b0-2b5036822fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:57.163313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc150d-4004-4508-a7b0-2b5036822fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:57.163788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc150d-4004-4508-a7b0-2b5036822fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:57.164270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc150d-4004-4508-a7b0-2b5036822fa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:57.204970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc2a49-5cfb-4e08-a7c3-88c8830a5921 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:57.236312 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc150d-4004-4508-a7b0-2b5036822fa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:57.238689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc150d-4004-4508-a7b0-2b5036822fa9 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:57.247557 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae140a8d-1a95-4432-b540-94bcd4a35cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:57.248262 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae140a8d-1a95-4432-b540-94bcd4a35cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:57.248482 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae140a8d-1a95-4432-b540-94bcd4a35cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:57.248748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae140a8d-1a95-4432-b540-94bcd4a35cf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:57.263600 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae140a8d-1a95-4432-b540-94bcd4a35cf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:57.265041 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae140a8d-1a95-4432-b540-94bcd4a35cf7 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:57.274286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:57.284634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-ae140a8d-1a95-4432-b540-94bcd4a35cf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:57.284634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:57.284634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:57.284634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:57.284634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:57.284634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:57.284634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:57.284634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:57.284634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:57.284634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:57.284634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:57.284634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:57.284634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:57.284634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:57.284634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:57.284634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:57.285897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:57.286963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:57.286963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:57.286963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:57.289100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 244/498] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:57.290751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47fc7e-4bca-4a38-b684-2ef990ed736a None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:57.291052 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47fc7e-4bca-4a38-b684-2ef990ed736a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:57.291453 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47fc7e-4bca-4a38-b684-2ef990ed736a None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:57.297246 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 245/499] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:57.298434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47fc7e-4bca-4a38-b684-2ef990ed736a None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:57.350043 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:57.383298 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 255/500] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:26:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 14:26:57.385058 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d078a-964c-4639-8f2a-3ae606f5b6e3 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:57.385515 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d078a-964c-4639-8f2a-3ae606f5b6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:57.386085 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d078a-964c-4639-8f2a-3ae606f5b6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:57.411396 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-f16d078a-964c-4639-8f2a-3ae606f5b6e3 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:57.413097 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-f16d078a-964c-4639-8f2a-3ae606f5b6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:57.425073 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47fc7e-4bca-4a38-b684-2ef990ed736a None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:57.464233 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46389ee-bea0-4a3d-96e3-d3ba12cbeb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:57.465017 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46389ee-bea0-4a3d-96e3-d3ba12cbeb71 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:57.465386 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46389ee-bea0-4a3d-96e3-d3ba12cbeb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:57.465750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46389ee-bea0-4a3d-96e3-d3ba12cbeb71 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:57.466496 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46389ee-bea0-4a3d-96e3-d3ba12cbeb71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:57.466887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46389ee-bea0-4a3d-96e3-d3ba12cbeb71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:57.468595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46389ee-bea0-4a3d-96e3-d3ba12cbeb71 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:57.482928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 246/501] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:57.483990 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46389ee-bea0-4a3d-96e3-d3ba12cbeb71 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:57.497907 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 256/502] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:57.499412 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d078a-964c-4639-8f2a-3ae606f5b6e3 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:57.601237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:57.685591 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46389ee-bea0-4a3d-96e3-d3ba12cbeb71 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:57.689472 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d078a-964c-4639-8f2a-3ae606f5b6e3 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:57.707421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:57.755368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f91a61-1d99-48af-9e60-7adab8975c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:57.755990 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f91a61-1d99-48af-9e60-7adab8975c51 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:57.756268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f91a61-1d99-48af-9e60-7adab8975c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:57.756538 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f91a61-1d99-48af-9e60-7adab8975c51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:57.759852 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a86ac0-0d61-4b7a-bf4c-e5dcf319a81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:57.760464 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a86ac0-0d61-4b7a-bf4c-e5dcf319a81d None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:57.760739 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a86ac0-0d61-4b7a-bf4c-e5dcf319a81d None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:57.761008 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a86ac0-0d61-4b7a-bf4c-e5dcf319a81d None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:57.761658 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a86ac0-0d61-4b7a-bf4c-e5dcf319a81d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:57.763090 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a86ac0-0d61-4b7a-bf4c-e5dcf319a81d None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:57.767233 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f91a61-1d99-48af-9e60-7adab8975c51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:57.769188 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f91a61-1d99-48af-9e60-7adab8975c51 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:57.794384 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 257/503] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:26:57 2022] POST /identity/v3/users => generated 254 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:26:57.796439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-29f91a61-1d99-48af-9e60-7adab8975c51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:57.796439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:57.796439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:57.796439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:57.796439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:57.796439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:57.796439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:57.796439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:57.796439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:57.796439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:57.796439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:57.796439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:57.796439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:57.796439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:57.796439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:57.796439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:57.797797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:57.798666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:57.798666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:57.798666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:57.798666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 247/504] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:57.803761 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a86ac0-0d61-4b7a-bf4c-e5dcf319a81d None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:57.835867 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f91a61-1d99-48af-9e60-7adab8975c51 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:58.003957 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a86ac0-0d61-4b7a-bf4c-e5dcf319a81d None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:58.011884 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f91a61-1d99-48af-9e60-7adab8975c51 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:58.024853 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:58.046780 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e4175-6ec6-4d9b-9748-94bc166f7e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:58.047607 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e4175-6ec6-4d9b-9748-94bc166f7e9c None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:58.047933 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e4175-6ec6-4d9b-9748-94bc166f7e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:58.048206 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e4175-6ec6-4d9b-9748-94bc166f7e9c None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:58.048860 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e4175-6ec6-4d9b-9748-94bc166f7e9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:58.049820 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e4175-6ec6-4d9b-9748-94bc166f7e9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:58.051271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e4175-6ec6-4d9b-9748-94bc166f7e9c None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:58.053866 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6548155-4f64-4fa8-bb34-5d11fb8420e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:58.054503 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6548155-4f64-4fa8-bb34-5d11fb8420e8 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:58.055387 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6548155-4f64-4fa8-bb34-5d11fb8420e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:58.055667 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6548155-4f64-4fa8-bb34-5d11fb8420e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:58.069774 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 258/505] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:58.072589 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6548155-4f64-4fa8-bb34-5d11fb8420e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:26:58.075544 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e4175-6ec6-4d9b-9748-94bc166f7e9c None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:58.078720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-a6548155-4f64-4fa8-bb34-5d11fb8420e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:26:58.078720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:58.078720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:58.078720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:58.078720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:58.078720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:58.078720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:58.078720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:58.078720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:58.078720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:58.078720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:58.078720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:58.078720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:26:58.078720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:26:58.078720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:26:58.078720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:26:58.079777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:26:58.079777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:26:58.079777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:26:58.079777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:26:58.079777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:58.079777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:58.079777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:58.079777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:58.079777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:58.079777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:58.079777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:58.079777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:58.079777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:58.079777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:58.079777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:58.079777 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:58.081128 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:58.081128 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:58.081128 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:58.081128 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:58.081128 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:26:58.081128 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:26:58.081128 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:26:58.081128 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:26:58.081128 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:26:58.081128 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:26:58.081128 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:26:58.081128 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:58.081128 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 248/506] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:26:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:58.089213 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6548155-4f64-4fa8-bb34-5d11fb8420e8 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:58.106821 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:58.233101 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e4175-6ec6-4d9b-9748-94bc166f7e9c None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:58.234308 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6548155-4f64-4fa8-bb34-5d11fb8420e8 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:58.293714 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7034bf8-d0fd-49fb-a2ff-b10ac4f530a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:58.294267 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7034bf8-d0fd-49fb-a2ff-b10ac4f530a6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:58.294539 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7034bf8-d0fd-49fb-a2ff-b10ac4f530a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:58.294805 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7034bf8-d0fd-49fb-a2ff-b10ac4f530a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:58.295349 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe40fa-ee7b-4b97-82b9-00f6201a321d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:58.296197 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe40fa-ee7b-4b97-82b9-00f6201a321d None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:58.296668 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe40fa-ee7b-4b97-82b9-00f6201a321d None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:58.297253 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe40fa-ee7b-4b97-82b9-00f6201a321d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:58.298465 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7034bf8-d0fd-49fb-a2ff-b10ac4f530a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:58.298786 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7034bf8-d0fd-49fb-a2ff-b10ac4f530a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:58.300652 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7034bf8-d0fd-49fb-a2ff-b10ac4f530a6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:58.306440 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe40fa-ee7b-4b97-82b9-00f6201a321d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:58.308279 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe40fa-ee7b-4b97-82b9-00f6201a321d None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:58.317541 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 249/507] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:26:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:58.324549 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7034bf8-d0fd-49fb-a2ff-b10ac4f530a6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:58.326446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-9bfe40fa-ee7b-4b97-82b9-00f6201a321d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:58.326446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:58.326446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:58.326446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:58.326446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:58.326446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:58.326446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:58.326446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:58.326446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:58.326446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:58.326446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:58.326446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:58.326446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:26:58.326446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:26:58.326446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:58.327436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:58.328318 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:26:58.328318 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:26:58.328318 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:26:58.328318 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:26:58.328318 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:26:58.328318 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:26:58.328318 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:26:58.328318 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:58.330629 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 259/508] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:58.339913 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe40fa-ee7b-4b97-82b9-00f6201a321d None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:58.384162 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:58.444976 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:58.459195 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7034bf8-d0fd-49fb-a2ff-b10ac4f530a6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:58.465215 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe40fa-ee7b-4b97-82b9-00f6201a321d None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:58.494136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e0b36-2820-4bc2-91ca-ee21c2f0b244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:58.494747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0b36-2820-4bc2-91ca-ee21c2f0b244 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:58.495028 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0b36-2820-4bc2-91ca-ee21c2f0b244 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:58.495319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0b36-2820-4bc2-91ca-ee21c2f0b244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:58.504650 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871c734-2a53-4660-ad36-7572fa6c21b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:58.505853 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871c734-2a53-4660-ad36-7572fa6c21b1 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:58.506329 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871c734-2a53-4660-ad36-7572fa6c21b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:58.506685 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e0b36-2820-4bc2-91ca-ee21c2f0b244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:58.507029 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871c734-2a53-4660-ad36-7572fa6c21b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:58.507936 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871c734-2a53-4660-ad36-7572fa6c21b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:58.508162 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e0b36-2820-4bc2-91ca-ee21c2f0b244 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:58.508662 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871c734-2a53-4660-ad36-7572fa6c21b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:58.510548 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871c734-2a53-4660-ad36-7572fa6c21b1 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:58.523066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-8d5e0b36-2820-4bc2-91ca-ee21c2f0b244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:58.523066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:58.523066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:58.523066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:58.523066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:58.523066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:58.523066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:58.523066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:58.523066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:58.523066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:58.523066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:58.523066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:58.523066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:58.523066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:58.523066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:58.523066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:58.524200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:58.525274 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:58.525274 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:58.525274 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:58.525274 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 250/509] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:58.531585 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 260/510] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:26:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:58.536117 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e0b36-2820-4bc2-91ca-ee21c2f0b244 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:58.537828 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871c734-2a53-4660-ad36-7572fa6c21b1 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:58.652969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871c734-2a53-4660-ad36-7572fa6c21b1 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:58.695691 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dfff5-4849-4e79-a44b-ff426ed5f019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:58.697419 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dfff5-4849-4e79-a44b-ff426ed5f019 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:58.698030 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dfff5-4849-4e79-a44b-ff426ed5f019 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:58.699601 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dfff5-4849-4e79-a44b-ff426ed5f019 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:58.700821 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:58.701766 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23dfff5-4849-4e79-a44b-ff426ed5f019 None admin] RBAC: Adding query filter params (role.id=745dcd2d05e447538a1aaff54b38193c, scope.project.id=720b11237d2e48128c171be45fcd0ad0, user.id=c8381a7c961047c69de894cf5b0055b6) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:58.703117 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23dfff5-4849-4e79-a44b-ff426ed5f019 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:58.706386 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23dfff5-4849-4e79-a44b-ff426ed5f019 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:58.723749 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 261/511] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:26:58 2022] GET /identity/v3/role_assignments?user.id=c8381a7c961047c69de894cf5b0055b6&scope.project.id=720b11237d2e48128c171be45fcd0ad0&role.id=745dcd2d05e447538a1aaff54b38193c => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:58.735806 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e0b36-2820-4bc2-91ca-ee21c2f0b244 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:58.788912 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e94b96-1af7-4b82-81d0-2403431b2721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:58.789853 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e94b96-1af7-4b82-81d0-2403431b2721 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:58.790311 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e94b96-1af7-4b82-81d0-2403431b2721 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:58.790760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e94b96-1af7-4b82-81d0-2403431b2721 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:58.791627 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e94b96-1af7-4b82-81d0-2403431b2721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:58.792134 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e94b96-1af7-4b82-81d0-2403431b2721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:58.793752 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e94b96-1af7-4b82-81d0-2403431b2721 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:58.795733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775de63f-4282-4f40-8579-ff589521a91e None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:58.796011 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775de63f-4282-4f40-8579-ff589521a91e None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:58.796281 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775de63f-4282-4f40-8579-ff589521a91e None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:58.797639 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 262/512] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:26:58.804208 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936379-d1bb-4a5f-823b-c6c963e0935c None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:58.804486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936379-d1bb-4a5f-823b-c6c963e0935c None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:58.804759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936379-d1bb-4a5f-823b-c6c963e0935c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:58.835040 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 251/513] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:58.836069 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-36936379-d1bb-4a5f-823b-c6c963e0935c None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:58.837279 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-36936379-d1bb-4a5f-823b-c6c963e0935c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:58.839483 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:58.841245 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e94b96-1af7-4b82-81d0-2403431b2721 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:58.983964 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 263/514] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:26:59.014028 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e94b96-1af7-4b82-81d0-2403431b2721 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:59.052264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce739d-a382-4b60-9c45-de02cb0b7bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:59.053165 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce739d-a382-4b60-9c45-de02cb0b7bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:59.053591 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce739d-a382-4b60-9c45-de02cb0b7bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:59.054010 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce739d-a382-4b60-9c45-de02cb0b7bc2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:59.062341 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce739d-a382-4b60-9c45-de02cb0b7bc2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:59.064057 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce739d-a382-4b60-9c45-de02cb0b7bc2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:59.077414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-64ce739d-a382-4b60-9c45-de02cb0b7bc2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 14:26:59.077414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:59.077414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:59.077414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:59.077414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:59.077414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:59.077414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:59.077414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:59.077414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:59.077414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:59.077414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:59.077414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:59.077414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:26:59.077414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:26:59.077414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:26:59.077414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:59.078559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:59.079737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:59.079737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:59.079737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:26:59.079737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:26:59.079737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:26:59.079737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:26:59.079737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:26:59.079737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:26:59.079737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 14:26:59.079737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:59.081986 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 252/515] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Oct 12 14:26:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:26:59.094523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36936379-d1bb-4a5f-823b-c6c963e0935c None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:59.217849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36936379-d1bb-4a5f-823b-c6c963e0935c None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:59.258539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f375d39-bef2-40a7-af30-644b7766cf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:59.259168 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f375d39-bef2-40a7-af30-644b7766cf62 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:59.259442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f375d39-bef2-40a7-af30-644b7766cf62 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:59.259709 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f375d39-bef2-40a7-af30-644b7766cf62 None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:59.260434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f375d39-bef2-40a7-af30-644b7766cf62 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:59.260732 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f375d39-bef2-40a7-af30-644b7766cf62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:59.262156 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f375d39-bef2-40a7-af30-644b7766cf62 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:59.274764 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:59.276053 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce739d-a382-4b60-9c45-de02cb0b7bc2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:59.280072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:59.286752 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 264/516] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Oct 12 14:26:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:59.292282 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f375d39-bef2-40a7-af30-644b7766cf62 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:59.425827 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f375d39-bef2-40a7-af30-644b7766cf62 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:59.430479 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce739d-a382-4b60-9c45-de02cb0b7bc2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:59.470218 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c496d8-1d44-4163-b9cd-e1c2f7e51ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:59.471037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b0282-2aa0-4a52-afa9-f516f5500594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:59.471260 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c496d8-1d44-4163-b9cd-e1c2f7e51ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:59.471341 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c496d8-1d44-4163-b9cd-e1c2f7e51ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:59.471608 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c496d8-1d44-4163-b9cd-e1c2f7e51ee7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:59.471718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b0282-2aa0-4a52-afa9-f516f5500594 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:59.471972 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b0282-2aa0-4a52-afa9-f516f5500594 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:59.472247 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b0282-2aa0-4a52-afa9-f516f5500594 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:59.472832 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b0282-2aa0-4a52-afa9-f516f5500594 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:59.474207 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b0282-2aa0-4a52-afa9-f516f5500594 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:59.483389 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c496d8-1d44-4163-b9cd-e1c2f7e51ee7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:59.484837 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c496d8-1d44-4163-b9cd-e1c2f7e51ee7 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-421b0282-2aa0-4a52-afa9-f516f5500594 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:59.485844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:59.487361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:59.487361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:59.487361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:59.487361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:59.487361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:59.487361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:59.487361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:26:59.487361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:26:59.487361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:26:59.487361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:26:59.487361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:59.487361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:59.487361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:59.487361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:59.487361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:59.487361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:59.488421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:59.488421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:59.488421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:59.488421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:59.488421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:59.488421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:59.488421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:59.488421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:59.488421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:59.488421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:59.488421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:26:59.488421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:26:59.488421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 14:26:59.488421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:26:59.488421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 253/517] 10.222.0.16 () {56 vars in 1129 bytes} [Wed Oct 12 14:26:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:59.489657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f3479-5b32-44c9-b310-8860628f75ee None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:59.489657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f3479-5b32-44c9-b310-8860628f75ee None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:59.489946 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f3479-5b32-44c9-b310-8860628f75ee None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:59.495197 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 254/518] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:26:59.496146 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f3479-5b32-44c9-b310-8860628f75ee None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:59.508259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-b5c496d8-1d44-4163-b9cd-e1c2f7e51ee7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:59.508259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:26:59.508259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:26:59.508259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:26:59.508259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:26:59.508259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:26:59.508259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:26:59.508259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:26:59.508259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:26:59.508259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:26:59.508259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:26:59.508259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:26:59.508259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:26:59.508259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:26:59.508259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:26:59.508259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:26:59.509545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:26:59.510405 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:26:59.510405 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:26:59.510405 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:26:59.511308 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 265/519] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:26:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:26:59.512913 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790137b7-f039-44e0-b5c3-ff41f9b609d4 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:59.513194 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790137b7-f039-44e0-b5c3-ff41f9b609d4 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:59.513464 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790137b7-f039-44e0-b5c3-ff41f9b609d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:59.532900 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-790137b7-f039-44e0-b5c3-ff41f9b609d4 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:26:59.534142 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-790137b7-f039-44e0-b5c3-ff41f9b609d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:26:59.583520 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:59.599318 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:59.620758 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 266/520] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:26:59.622148 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790137b7-f039-44e0-b5c3-ff41f9b609d4 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:59.655302 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f3479-5b32-44c9-b310-8860628f75ee None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:59.696006 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a058c-5682-422f-974d-bfc49d01d50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:59.696961 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a058c-5682-422f-974d-bfc49d01d50e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:59.697437 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a058c-5682-422f-974d-bfc49d01d50e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:59.698038 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a058c-5682-422f-974d-bfc49d01d50e None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:59.698868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a058c-5682-422f-974d-bfc49d01d50e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:59.700206 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a058c-5682-422f-974d-bfc49d01d50e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:59.701822 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a058c-5682-422f-974d-bfc49d01d50e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:59.707715 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 255/521] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Oct 12 14:26:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:59.716581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a058c-5682-422f-974d-bfc49d01d50e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:59.833490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790137b7-f039-44e0-b5c3-ff41f9b609d4 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:59.903326 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a058c-5682-422f-974d-bfc49d01d50e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:26:59.904289 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0eea92-34f2-49fd-894a-5315b4322be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:59.904900 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0eea92-34f2-49fd-894a-5315b4322be8 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:59.905168 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0eea92-34f2-49fd-894a-5315b4322be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:59.905435 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0eea92-34f2-49fd-894a-5315b4322be8 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:59.906073 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0eea92-34f2-49fd-894a-5315b4322be8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:59.906369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0eea92-34f2-49fd-894a-5315b4322be8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:59.907795 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0eea92-34f2-49fd-894a-5315b4322be8 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:59.931982 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 267/522] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:26:59.937287 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0eea92-34f2-49fd-894a-5315b4322be8 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:59.962192 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:26:59.967799 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3384a6f-7d60-41a0-9a07-189e05adfee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:26:59.968412 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3384a6f-7d60-41a0-9a07-189e05adfee1 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:26:59.968683 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3384a6f-7d60-41a0-9a07-189e05adfee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:26:59.968950 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3384a6f-7d60-41a0-9a07-189e05adfee1 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:26:59.969578 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3384a6f-7d60-41a0-9a07-189e05adfee1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:26:59.969878 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3384a6f-7d60-41a0-9a07-189e05adfee1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:26:59.971792 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3384a6f-7d60-41a0-9a07-189e05adfee1 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:26:59.977877 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 256/523] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Oct 12 14:26:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:26:59.978664 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3384a6f-7d60-41a0-9a07-189e05adfee1 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:26:59.993441 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:00.097659 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0eea92-34f2-49fd-894a-5315b4322be8 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:00.137006 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3384a6f-7d60-41a0-9a07-189e05adfee1 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:00.179341 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f28fd8-e2e9-43c7-8d2c-2cfd29391fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:00.180307 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f28fd8-e2e9-43c7-8d2c-2cfd29391fbc None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:00.180766 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f28fd8-e2e9-43c7-8d2c-2cfd29391fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:00.181215 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f28fd8-e2e9-43c7-8d2c-2cfd29391fbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:00.195792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f28fd8-e2e9-43c7-8d2c-2cfd29391fbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:00.197963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f28fd8-e2e9-43c7-8d2c-2cfd29391fbc None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:00.204773 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb43ddc-df47-4b8f-af68-5522a1177ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:00.205952 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb43ddc-df47-4b8f-af68-5522a1177ace None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:00.206229 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb43ddc-df47-4b8f-af68-5522a1177ace None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:00.206517 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb43ddc-df47-4b8f-af68-5522a1177ace None admin] PATH_INFO: `/v3/services/network` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:00.207074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb43ddc-df47-4b8f-af68-5522a1177ace None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:00.208526 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb43ddc-df47-4b8f-af68-5522a1177ace None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:00.215484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-13f28fd8-e2e9-43c7-8d2c-2cfd29391fbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:00.215484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:00.215484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:00.215484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:00.215484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:00.215484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:00.215484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:00.215484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:00.215484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:00.215484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:00.215484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:00.215484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:00.215484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:27:00.215484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:27:00.215484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:00.216345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:00.217206 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:27:00.217206 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:27:00.217206 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:27:00.217206 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:27:00.217206 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:27:00.217206 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:27:00.217206 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:00.217206 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-6eb43ddc-df47-4b8f-af68-5522a1177ace None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:00.218590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:00.219572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:00.219572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:00.219572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:00.219572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:00.219572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:00.219572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:00.219572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:00.219572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:00.219572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:00.219572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:00.219572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:00.219572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:00.219572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:00.219572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:00.219572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:00.219572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:00.220424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:00.220424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:00.220424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:00.220424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:00.220424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:00.220424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:00.220424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:00.220424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:00.220424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:00.220424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:00.220424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:00.220424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:00.220424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 14:27:00.220424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:00.221405 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 257/524] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:59 2022] GET /identity/v3/services/network => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:00.222903 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 268/525] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:26:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:00.226297 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb43ddc-df47-4b8f-af68-5522a1177ace None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:00.227946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f28fd8-e2e9-43c7-8d2c-2cfd29391fbc None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:00.304035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:00.388564 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb43ddc-df47-4b8f-af68-5522a1177ace None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:00.408832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:00.415336 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f28fd8-e2e9-43c7-8d2c-2cfd29391fbc None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:00.440321 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677b674-e3d1-455a-b939-838593cc44d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:00.441135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677b674-e3d1-455a-b939-838593cc44d5 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:00.441496 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677b674-e3d1-455a-b939-838593cc44d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:00.441866 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677b674-e3d1-455a-b939-838593cc44d5 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:00.442620 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677b674-e3d1-455a-b939-838593cc44d5 None admin] RBAC: Adding query filter params (name=network) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:00.446154 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677b674-e3d1-455a-b939-838593cc44d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:00.446154 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677b674-e3d1-455a-b939-838593cc44d5 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:00.450473 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 258/526] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Oct 12 14:27:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:00.456472 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677b674-e3d1-455a-b939-838593cc44d5 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:00.468720 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbf24a-4780-4667-a49e-30c2d321feb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:00.469624 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbf24a-4780-4667-a49e-30c2d321feb9 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:00.470077 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbf24a-4780-4667-a49e-30c2d321feb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:00.470522 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbf24a-4780-4667-a49e-30c2d321feb9 None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:00.471929 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dbf24a-4780-4667-a49e-30c2d321feb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:00.472410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dbf24a-4780-4667-a49e-30c2d321feb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:00.474008 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dbf24a-4780-4667-a49e-30c2d321feb9 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:00.486911 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 269/527] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:27:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:00.492819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbf24a-4780-4667-a49e-30c2d321feb9 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:00.603045 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677b674-e3d1-455a-b939-838593cc44d5 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:00.611499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbf24a-4780-4667-a49e-30c2d321feb9 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:00.634745 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:00.645313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b8488-2f36-4386-813a-ce91458a51fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:00.646370 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b8488-2f36-4386-813a-ce91458a51fb None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:00.646712 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b8488-2f36-4386-813a-ce91458a51fb None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:00.646991 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b8488-2f36-4386-813a-ce91458a51fb None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:00.647933 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b8488-2f36-4386-813a-ce91458a51fb None admin] RBAC: Adding query filter params (type=network) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:00.648222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b8488-2f36-4386-813a-ce91458a51fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:00.650564 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b8488-2f36-4386-813a-ce91458a51fb None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:00.655135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07c3f0-9f3c-445e-9f6a-bb82aa89c941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:00.655772 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 259/528] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Oct 12 14:27:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:00.656173 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07c3f0-9f3c-445e-9f6a-bb82aa89c941 None admin] REQUEST_METHOD: `PUT` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:00.656584 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07c3f0-9f3c-445e-9f6a-bb82aa89c941 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:00.656948 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07c3f0-9f3c-445e-9f6a-bb82aa89c941 None admin] PATH_INFO: `/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/d0aeed5f32a1477dbd9d3e934669705b/roles/4b1f48ab911e4fadbc70164ae43d508b` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:00.657061 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88172c8-5a75-4cfd-8665-6b5da38e298c None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:00.657352 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88172c8-5a75-4cfd-8665-6b5da38e298c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:00.657631 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88172c8-5a75-4cfd-8665-6b5da38e298c None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:00.659923 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 260/529] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:00.663981 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07c3f0-9f3c-445e-9f6a-bb82aa89c941 None admin] RBAC: Authorizing `identity:create_grant(project_id=720b11237d2e48128c171be45fcd0ad0, user_id=d0aeed5f32a1477dbd9d3e934669705b, role_id=4b1f48ab911e4fadbc70164ae43d508b)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:00.665461 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07c3f0-9f3c-445e-9f6a-bb82aa89c941 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:00.668261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f6e83-05ce-4a2a-811e-830a60668015 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:00.668532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f6e83-05ce-4a2a-811e-830a60668015 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:00.668806 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f6e83-05ce-4a2a-811e-830a60668015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:00.690204 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 270/530] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:27:00 2022] PUT /identity/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/d0aeed5f32a1477dbd9d3e934669705b/roles/4b1f48ab911e4fadbc70164ae43d508b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:27:00.703748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d64f6e83-05ce-4a2a-811e-830a60668015 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:00.705322 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d64f6e83-05ce-4a2a-811e-830a60668015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:00.811598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 261/531] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:27:01.129952 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07c3f0-9f3c-445e-9f6a-bb82aa89c941 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:01.155097 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:01.259711 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07c3f0-9f3c-445e-9f6a-bb82aa89c941 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:01.296006 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1bb2f1-2a92-4e5f-b249-11225f30b671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:01.296641 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bb2f1-2a92-4e5f-b249-11225f30b671 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:01.296917 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bb2f1-2a92-4e5f-b249-11225f30b671 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:01.297184 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bb2f1-2a92-4e5f-b249-11225f30b671 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:01.308326 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1bb2f1-2a92-4e5f-b249-11225f30b671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:27:01.313468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-7d1bb2f1-2a92-4e5f-b249-11225f30b671 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:27:01.313468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:01.313468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:01.313468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:01.313468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:01.313468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:01.313468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:01.313468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:01.313468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:01.313468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:01.313468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:01.313468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:01.313468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:27:01.313468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:27:01.313468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:27:01.313468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:27:01.314367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:27:01.314367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:27:01.314367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:27:01.314367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:27:01.314367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:01.314367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:01.314367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:01.314367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:01.314367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:01.314367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:01.314367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:01.314367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:01.314367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:01.314367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:01.314367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:01.314367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:01.316143 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:01.316143 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:01.316143 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:01.316143 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:01.316143 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:27:01.316143 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:27:01.316143 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:27:01.316143 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:27:01.316143 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:27:01.316143 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:27:01.316143 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:27:01.316143 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:01.316143 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 271/532] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Oct 12 14:27:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:01.323022 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f6e83-05ce-4a2a-811e-830a60668015 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:01.339439 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5d0b7-394d-4f8c-85a3-1199c1c081d2 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:01.339743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5d0b7-394d-4f8c-85a3-1199c1c081d2 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:01.339965 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5d0b7-394d-4f8c-85a3-1199c1c081d2 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:01.344669 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 272/533] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:01.356043 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd4a98-65da-42da-8490-e2d5ca7bcff8 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:01.356315 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd4a98-65da-42da-8490-e2d5ca7bcff8 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:01.356585 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd4a98-65da-42da-8490-e2d5ca7bcff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:01.393096 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-eebd4a98-65da-42da-8490-e2d5ca7bcff8 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:01.394368 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-eebd4a98-65da-42da-8490-e2d5ca7bcff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:01.484457 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f6e83-05ce-4a2a-811e-830a60668015 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:01.502503 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:01.527111 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8816d-fbc1-4eff-923f-ae77854a288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:01.527748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8816d-fbc1-4eff-923f-ae77854a288a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:01.528020 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8816d-fbc1-4eff-923f-ae77854a288a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:01.528290 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8816d-fbc1-4eff-923f-ae77854a288a None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:01.528958 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8816d-fbc1-4eff-923f-ae77854a288a None admin] RBAC: Adding query filter params (name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:01.529255 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8816d-fbc1-4eff-923f-ae77854a288a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:01.535096 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8816d-fbc1-4eff-923f-ae77854a288a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:01.554151 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 262/534] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Oct 12 14:27:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:01.561726 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 273/535] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 14:27:01.562875 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8816d-fbc1-4eff-923f-ae77854a288a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:01.708976 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8816d-fbc1-4eff-923f-ae77854a288a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:01.755939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b7636-a1dd-4695-bb18-3be14ffffd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:01.756575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b7636-a1dd-4695-bb18-3be14ffffd2c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:01.756848 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b7636-a1dd-4695-bb18-3be14ffffd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:01.757116 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b7636-a1dd-4695-bb18-3be14ffffd2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:01.769706 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b7636-a1dd-4695-bb18-3be14ffffd2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:01.771478 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b7636-a1dd-4695-bb18-3be14ffffd2c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:01.794632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-a04b7636-a1dd-4695-bb18-3be14ffffd2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:01.794632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:01.794632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:01.794632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:01.794632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:01.794632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:01.794632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:01.794632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:01.794632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:01.794632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:01.794632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:01.794632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:01.794632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:01.794632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:01.794632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:01.794632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:01.795649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:01.796509 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:01.796509 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:01.796509 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:01.800634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 263/536] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:01.806039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd4a98-65da-42da-8490-e2d5ca7bcff8 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:01.825471 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:01.954043 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd4a98-65da-42da-8490-e2d5ca7bcff8 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:01.959364 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b7636-a1dd-4695-bb18-3be14ffffd2c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:01.998616 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:02.006909 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52012d-afda-45c1-bb47-a37053157305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:02.007545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52012d-afda-45c1-bb47-a37053157305 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:02.007820 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52012d-afda-45c1-bb47-a37053157305 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:02.008088 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52012d-afda-45c1-bb47-a37053157305 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:02.008734 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52012d-afda-45c1-bb47-a37053157305 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:02.009034 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52012d-afda-45c1-bb47-a37053157305 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:02.010408 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52012d-afda-45c1-bb47-a37053157305 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:02.042073 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 274/537] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:02.044560 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52012d-afda-45c1-bb47-a37053157305 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:02.129934 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b7636-a1dd-4695-bb18-3be14ffffd2c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:02.170680 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:02.196687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26432174-65ae-4b0c-9590-189aae7f74ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:02.197363 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26432174-65ae-4b0c-9590-189aae7f74ab None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:02.197638 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26432174-65ae-4b0c-9590-189aae7f74ab None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:02.197904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26432174-65ae-4b0c-9590-189aae7f74ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:02.213293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26432174-65ae-4b0c-9590-189aae7f74ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:27:02.220749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-26432174-65ae-4b0c-9590-189aae7f74ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:27:02.220749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:02.220749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:02.220749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:02.220749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:02.220749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:02.220749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:02.220749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:02.220749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:02.220749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:02.220749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:02.220749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:02.220749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:27:02.220749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:27:02.220749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:27:02.220749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:27:02.222391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:27:02.222391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:27:02.222391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:27:02.222391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:27:02.222391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:02.222391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:02.222391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:02.222391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:02.222391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:02.222391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:02.222391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:02.222391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:02.222391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:02.222391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:02.222391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:02.222391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:02.223342 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:02.223342 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:02.223342 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:02.223342 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:02.223342 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:27:02.223342 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:27:02.223342 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:27:02.223342 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:27:02.223342 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:27:02.223342 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:27:02.223342 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:27:02.223342 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:02.223961 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 264/538] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:27:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:02.230068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26432174-65ae-4b0c-9590-189aae7f74ab None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:02.231700 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52012d-afda-45c1-bb47-a37053157305 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:02.294136 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718959ed-cbbd-4cd7-a8b3-bf95a5f2e4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:02.294774 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718959ed-cbbd-4cd7-a8b3-bf95a5f2e4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:02.295051 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718959ed-cbbd-4cd7-a8b3-bf95a5f2e4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:02.295347 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718959ed-cbbd-4cd7-a8b3-bf95a5f2e4f8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:02.313893 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718959ed-cbbd-4cd7-a8b3-bf95a5f2e4f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:02.315713 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718959ed-cbbd-4cd7-a8b3-bf95a5f2e4f8 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:02.323989 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:02.333528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26432174-65ae-4b0c-9590-189aae7f74ab None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:02.334546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-718959ed-cbbd-4cd7-a8b3-bf95a5f2e4f8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 14:27:02.334546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:02.334546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:02.334546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:02.334546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:02.334546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:02.334546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:02.334546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:02.334546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:02.334546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:02.334546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:02.334546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:02.334546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:27:02.334546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:27:02.334546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:27:02.334546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:02.335499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:02.336400 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:02.336400 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:02.336400 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:27:02.336400 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:27:02.336400 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:27:02.336400 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:27:02.336400 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:27:02.336400 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:27:02.336400 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 14:27:02.336400 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:02.343582 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 275/539] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Oct 12 14:27:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:02.349414 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718959ed-cbbd-4cd7-a8b3-bf95a5f2e4f8 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:02.369707 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a80fc-042e-4ac1-a74c-6cca841aefb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:02.370316 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a80fc-042e-4ac1-a74c-6cca841aefb7 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:02.370586 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a80fc-042e-4ac1-a74c-6cca841aefb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:02.370853 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a80fc-042e-4ac1-a74c-6cca841aefb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:02.371549 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a80fc-042e-4ac1-a74c-6cca841aefb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:02.371844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a80fc-042e-4ac1-a74c-6cca841aefb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:02.373273 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a80fc-042e-4ac1-a74c-6cca841aefb7 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:02.382266 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 265/540] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:27:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:02.390120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a80fc-042e-4ac1-a74c-6cca841aefb7 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:02.508826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a80fc-042e-4ac1-a74c-6cca841aefb7 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:02.521325 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718959ed-cbbd-4cd7-a8b3-bf95a5f2e4f8 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:02.545570 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3ef03-91d3-4be4-b601-1a22ad2f0f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:02.546443 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3ef03-91d3-4be4-b601-1a22ad2f0f7f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:02.546896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3ef03-91d3-4be4-b601-1a22ad2f0f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:02.547358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3ef03-91d3-4be4-b601-1a22ad2f0f7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:02.558852 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3ef03-91d3-4be4-b601-1a22ad2f0f7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:02.560599 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3ef03-91d3-4be4-b601-1a22ad2f0f7f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:02.574114 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:02.576751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-8bb3ef03-91d3-4be4-b601-1a22ad2f0f7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:02.576751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:02.576751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:02.576751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:02.576751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:02.576751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:02.576751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:02.576751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:02.576751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:02.576751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:02.576751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:02.576751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:02.576751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:02.576751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:02.576751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:02.576751 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:02.578522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:02.579546 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:02.579546 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:02.579546 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:02.581209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 266/541] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:02.591818 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3ef03-91d3-4be4-b601-1a22ad2f0f7f None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:02.602175 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb309f9d-0407-4166-aae7-2db5bf6b0f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:02.602787 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb309f9d-0407-4166-aae7-2db5bf6b0f2b None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:02.603061 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb309f9d-0407-4166-aae7-2db5bf6b0f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:02.603361 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb309f9d-0407-4166-aae7-2db5bf6b0f2b None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:02.604091 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb309f9d-0407-4166-aae7-2db5bf6b0f2b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:02.604390 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb309f9d-0407-4166-aae7-2db5bf6b0f2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:02.605813 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb309f9d-0407-4166-aae7-2db5bf6b0f2b None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:02.621396 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:02.636877 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 276/542] 10.222.0.16 () {56 vars in 1149 bytes} [Wed Oct 12 14:27:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:02.639182 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffb0be-45b7-4f51-9447-5fc9daaec547 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:02.639509 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffb0be-45b7-4f51-9447-5fc9daaec547 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:02.639784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffb0be-45b7-4f51-9447-5fc9daaec547 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:02.641239 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 277/543] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:02.643579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffb0be-45b7-4f51-9447-5fc9daaec547 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:02.740068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3ef03-91d3-4be4-b601-1a22ad2f0f7f None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:02.754633 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffb0be-45b7-4f51-9447-5fc9daaec547 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:02.782941 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335c015-3322-414e-bab9-8581983b7cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:02.784505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335c015-3322-414e-bab9-8581983b7cbf None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:02.784990 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335c015-3322-414e-bab9-8581983b7cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:02.785265 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335c015-3322-414e-bab9-8581983b7cbf None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:02.786504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335c015-3322-414e-bab9-8581983b7cbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:02.786813 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335c015-3322-414e-bab9-8581983b7cbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:02.788380 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335c015-3322-414e-bab9-8581983b7cbf None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:02.793635 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e36db-7739-4bbe-84a2-e0feae233418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:02.794381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e36db-7739-4bbe-84a2-e0feae233418 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:02.794750 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e36db-7739-4bbe-84a2-e0feae233418 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:02.795181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e36db-7739-4bbe-84a2-e0feae233418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:02.803231 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 267/544] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:02.804538 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe612d-561f-472c-a45b-6ecec9752e08 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:02.804818 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe612d-561f-472c-a45b-6ecec9752e08 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:02.805092 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe612d-561f-472c-a45b-6ecec9752e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:02.811852 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e36db-7739-4bbe-84a2-e0feae233418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:02.813304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e36db-7739-4bbe-84a2-e0feae233418 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:02.829992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3abe612d-561f-472c-a45b-6ecec9752e08 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:02.831563 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3abe612d-561f-472c-a45b-6ecec9752e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:02.838874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-584e36db-7739-4bbe-84a2-e0feae233418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:02.838874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:02.838874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:02.838874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:02.838874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:02.838874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:02.838874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:02.838874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:02.838874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:02.838874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:02.838874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:02.838874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:02.838874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:02.838874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:02.838874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:02.838874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:02.839840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:02.840704 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:02.840704 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:02.840704 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:02.848089 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 278/545] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:02.849130 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e36db-7739-4bbe-84a2-e0feae233418 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:02.892724 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:02.899349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:02.907674 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 268/546] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:27:02.912801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe612d-561f-472c-a45b-6ecec9752e08 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:02.961801 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e36db-7739-4bbe-84a2-e0feae233418 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:02.998163 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5e246-1ac4-49f8-af0b-e1621ddb9cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:02.999160 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5e246-1ac4-49f8-af0b-e1621ddb9cfa None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:02.999674 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5e246-1ac4-49f8-af0b-e1621ddb9cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:03.000137 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5e246-1ac4-49f8-af0b-e1621ddb9cfa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:03.008126 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5e246-1ac4-49f8-af0b-e1621ddb9cfa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:03.009873 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5e246-1ac4-49f8-af0b-e1621ddb9cfa None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:03.022747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-91b5e246-1ac4-49f8-af0b-e1621ddb9cfa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 12 14:27:03.022747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:03.022747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:03.022747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:03.022747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:03.022747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:03.022747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:03.022747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:03.022747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:03.022747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:03.022747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:03.022747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:03.022747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:27:03.022747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:27:03.022747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:27:03.022747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:03.023760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:03.024762 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:03.024762 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:03.024762 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:27:03.024762 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:27:03.024762 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:27:03.024762 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:27:03.024762 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:27:03.024762 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:27:03.024762 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 12 14:27:03.024762 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:03.031254 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 279/547] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:27:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:03.033070 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5e246-1ac4-49f8-af0b-e1621ddb9cfa None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:03.113935 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe612d-561f-472c-a45b-6ecec9752e08 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:03.135645 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:03.140405 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5e246-1ac4-49f8-af0b-e1621ddb9cfa None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:03.177559 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ec1a9-9bc4-4d3e-af5a-c6b6f7733912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:03.178512 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ec1a9-9bc4-4d3e-af5a-c6b6f7733912 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:03.179011 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ec1a9-9bc4-4d3e-af5a-c6b6f7733912 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:03.179496 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ec1a9-9bc4-4d3e-af5a-c6b6f7733912 None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:03.180704 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ec1a9-9bc4-4d3e-af5a-c6b6f7733912 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:03.181304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ec1a9-9bc4-4d3e-af5a-c6b6f7733912 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:03.182950 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ec1a9-9bc4-4d3e-af5a-c6b6f7733912 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:03.184766 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896463d2-1ef0-4dec-885a-eda43db96562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:03.185436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896463d2-1ef0-4dec-885a-eda43db96562 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:03.185613 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896463d2-1ef0-4dec-885a-eda43db96562 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:03.185881 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896463d2-1ef0-4dec-885a-eda43db96562 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:03.186522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896463d2-1ef0-4dec-885a-eda43db96562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:03.186814 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896463d2-1ef0-4dec-885a-eda43db96562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:03.188204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896463d2-1ef0-4dec-885a-eda43db96562 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:03.203230 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 269/548] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:03.213028 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896463d2-1ef0-4dec-885a-eda43db96562 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:03.213378 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 280/549] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Oct 12 14:27:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:03.215226 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ec1a9-9bc4-4d3e-af5a-c6b6f7733912 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:03.334168 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ec1a9-9bc4-4d3e-af5a-c6b6f7733912 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:03.337308 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:03.369741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b27b1-8384-4702-98eb-04ce33bef493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:03.370367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b27b1-8384-4702-98eb-04ce33bef493 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:03.370641 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b27b1-8384-4702-98eb-04ce33bef493 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:03.370910 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b27b1-8384-4702-98eb-04ce33bef493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:03.381595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b27b1-8384-4702-98eb-04ce33bef493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:03.383035 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b27b1-8384-4702-98eb-04ce33bef493 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:03.388129 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896463d2-1ef0-4dec-885a-eda43db96562 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:03.398599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-957b27b1-8384-4702-98eb-04ce33bef493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:03.398599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:03.398599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:03.398599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:03.398599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:03.398599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:03.398599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:03.398599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:03.398599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:03.398599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:03.398599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:03.398599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:03.398599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:03.398599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:03.398599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:03.398599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:03.399818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:03.400947 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:03.400947 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:03.400947 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:03.400947 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 281/550] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:03.400947 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b27b1-8384-4702-98eb-04ce33bef493 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:03.411849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:03.429203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22780c-ccf4-4aca-aa5f-595963906293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:03.429896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22780c-ccf4-4aca-aa5f-595963906293 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:03.430234 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22780c-ccf4-4aca-aa5f-595963906293 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:03.430562 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22780c-ccf4-4aca-aa5f-595963906293 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:03.437016 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22780c-ccf4-4aca-aa5f-595963906293 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:03.438449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22780c-ccf4-4aca-aa5f-595963906293 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:03.449471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-da22780c-ccf4-4aca-aa5f-595963906293 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:03.449471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:03.449471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:03.449471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:03.449471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:03.449471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:03.449471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:03.449471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:03.449471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:03.449471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:03.449471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:03.449471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:03.449471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:27:03.449471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:27:03.449471 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:03.450334 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:03.451256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:27:03.451256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:27:03.451256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:27:03.451256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:27:03.451256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:27:03.451256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:27:03.451256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:03.451256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:03.451256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 270/551] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:27:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:03.452014 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53582655-a842-48c0-ad21-a4da04c52f9b None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:03.452014 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53582655-a842-48c0-ad21-a4da04c52f9b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:03.452252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53582655-a842-48c0-ad21-a4da04c52f9b None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:03.453373 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 271/552] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:03.454363 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fc16f-7e88-45b2-82ab-34adc771c395 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:03.454635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fc16f-7e88-45b2-82ab-34adc771c395 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:03.454914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fc16f-7e88-45b2-82ab-34adc771c395 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:03.456584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 272/553] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:03.457204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fc16f-7e88-45b2-82ab-34adc771c395 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:03.510716 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b27b1-8384-4702-98eb-04ce33bef493 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:03.544290 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980913c-fdfe-41b3-9476-0e305f38e9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:03.544900 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980913c-fdfe-41b3-9476-0e305f38e9f5 None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:03.545159 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980913c-fdfe-41b3-9476-0e305f38e9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:03.545420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980913c-fdfe-41b3-9476-0e305f38e9f5 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:03.546055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980913c-fdfe-41b3-9476-0e305f38e9f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:03.547467 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980913c-fdfe-41b3-9476-0e305f38e9f5 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:03.559403 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fc16f-7e88-45b2-82ab-34adc771c395 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:03.563418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 282/554] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Oct 12 14:27:03 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:27:03.564626 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980913c-fdfe-41b3-9476-0e305f38e9f5 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:03.577033 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:03.613803 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d84ff1-689d-42e9-871c-72a4310d4ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:03.614333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d84ff1-689d-42e9-871c-72a4310d4ecf None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:03.614612 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d84ff1-689d-42e9-871c-72a4310d4ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:03.614877 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d84ff1-689d-42e9-871c-72a4310d4ecf None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:03.615582 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d84ff1-689d-42e9-871c-72a4310d4ecf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:03.615881 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d84ff1-689d-42e9-871c-72a4310d4ecf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:03.617306 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d84ff1-689d-42e9-871c-72a4310d4ecf None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:03.631227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 273/555] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:03.632545 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fffba9-ba21-4057-aa89-9cf68520b441 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:03.632828 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fffba9-ba21-4057-aa89-9cf68520b441 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:03.633100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fffba9-ba21-4057-aa89-9cf68520b441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:03.652950 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-09fffba9-ba21-4057-aa89-9cf68520b441 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:03.654187 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-09fffba9-ba21-4057-aa89-9cf68520b441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:03.668540 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:03.710455 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980913c-fdfe-41b3-9476-0e305f38e9f5 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:03.729813 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 274/556] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:27:03.731699 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b051858-acb6-4ed2-8422-e346469b3d9e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:03.732067 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b051858-acb6-4ed2-8422-e346469b3d9e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:03.732391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b051858-acb6-4ed2-8422-e346469b3d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:03.748441 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e295f43-3f3a-458b-9e5e-406f8dd9e74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:03.749351 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e295f43-3f3a-458b-9e5e-406f8dd9e74f None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:03.749639 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e295f43-3f3a-458b-9e5e-406f8dd9e74f None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:03.749887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e295f43-3f3a-458b-9e5e-406f8dd9e74f None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:03.750606 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e295f43-3f3a-458b-9e5e-406f8dd9e74f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:03.750909 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e295f43-3f3a-458b-9e5e-406f8dd9e74f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:03.753276 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e295f43-3f3a-458b-9e5e-406f8dd9e74f None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:03.764244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-6b051858-acb6-4ed2-8422-e346469b3d9e None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:03.765200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6b051858-acb6-4ed2-8422-e346469b3d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:03.765724 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 283/557] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:27:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:03.766332 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e295f43-3f3a-458b-9e5e-406f8dd9e74f None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:03.849918 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 275/558] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 14:27:03.850888 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b051858-acb6-4ed2-8422-e346469b3d9e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:03.892075 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e295f43-3f3a-458b-9e5e-406f8dd9e74f None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:03.919096 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:03.926716 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:03.946541 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca91cf9-1f32-42e6-a274-8f8030d24af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:03.947191 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca91cf9-1f32-42e6-a274-8f8030d24af3 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:03.947469 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca91cf9-1f32-42e6-a274-8f8030d24af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:03.947734 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca91cf9-1f32-42e6-a274-8f8030d24af3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:03.962720 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca91cf9-1f32-42e6-a274-8f8030d24af3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:03.964874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca91cf9-1f32-42e6-a274-8f8030d24af3 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:03.984622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-2ca91cf9-1f32-42e6-a274-8f8030d24af3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:03.984622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:03.984622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:03.984622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:03.984622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:03.984622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:03.984622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:03.984622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:03.984622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:03.984622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:03.984622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:03.984622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:03.984622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:27:03.984622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:27:03.984622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:03.985528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:03.986465 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:27:03.986465 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:27:03.986465 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:27:03.986465 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:27:03.986465 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:27:03.986465 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:27:03.986465 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:03.986465 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:03.988434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 284/559] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:27:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:03.999838 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca91cf9-1f32-42e6-a274-8f8030d24af3 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:04.028516 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b051858-acb6-4ed2-8422-e346469b3d9e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:04.078112 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86362b-f8a0-4a7d-8ff8-44e0e8418924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:04.079174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86362b-f8a0-4a7d-8ff8-44e0e8418924 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:04.079666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86362b-f8a0-4a7d-8ff8-44e0e8418924 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:04.080147 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86362b-f8a0-4a7d-8ff8-44e0e8418924 None admin] PATH_INFO: `/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/15d7cb817428475e80cf697ac94c62b2/roles/745dcd2d05e447538a1aaff54b38193c` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:04.098818 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86362b-f8a0-4a7d-8ff8-44e0e8418924 None admin] RBAC: Authorizing `identity:create_grant(project_id=720b11237d2e48128c171be45fcd0ad0, user_id=15d7cb817428475e80cf697ac94c62b2, role_id=745dcd2d05e447538a1aaff54b38193c)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:04.102375 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86362b-f8a0-4a7d-8ff8-44e0e8418924 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:04.120564 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 276/560] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:27:03 2022] PUT /identity/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/15d7cb817428475e80cf697ac94c62b2/roles/745dcd2d05e447538a1aaff54b38193c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:27:04.121030 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca91cf9-1f32-42e6-a274-8f8030d24af3 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:04.122063 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86362b-f8a0-4a7d-8ff8-44e0e8418924 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:04.215802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6a50b-536c-4f1f-951a-63de640e2a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:04.216422 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6a50b-536c-4f1f-951a-63de640e2a90 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:04.216691 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6a50b-536c-4f1f-951a-63de640e2a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:04.216951 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6a50b-536c-4f1f-951a-63de640e2a90 None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:04.217645 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd6a50b-536c-4f1f-951a-63de640e2a90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:04.217942 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd6a50b-536c-4f1f-951a-63de640e2a90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:04.219504 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd6a50b-536c-4f1f-951a-63de640e2a90 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:04.235216 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 285/561] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:27:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:04.235216 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6a50b-536c-4f1f-951a-63de640e2a90 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:04.264653 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:04.280752 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86362b-f8a0-4a7d-8ff8-44e0e8418924 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:04.312797 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:04.328316 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dbedf-789d-4779-8d6e-12f28b47f203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:04.328953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dbedf-789d-4779-8d6e-12f28b47f203 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:04.329231 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dbedf-789d-4779-8d6e-12f28b47f203 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:04.329650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dbedf-789d-4779-8d6e-12f28b47f203 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:04.342758 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8dbedf-789d-4779-8d6e-12f28b47f203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:27:04.350362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-fe8dbedf-789d-4779-8d6e-12f28b47f203 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:27:04.350362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:04.350362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:04.350362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:04.350362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:04.350362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:04.350362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:04.350362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:04.350362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:04.350362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:04.350362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:04.350362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:04.350362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:27:04.350362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:27:04.350362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:27:04.350362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:27:04.351384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:27:04.351384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:27:04.351384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:27:04.351384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:27:04.351384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:04.351384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:04.351384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:04.351384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:04.351384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:04.351384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:04.351384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:04.351384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:04.351384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:04.351384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:04.351384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:04.351384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:04.352212 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:04.352212 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:04.352212 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:04.352212 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:04.352212 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:27:04.352212 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:27:04.352212 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:27:04.352212 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:27:04.352212 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:27:04.352212 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:27:04.352212 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:27:04.352212 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:04.352212 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 277/562] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:27:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:04.352882 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dbedf-789d-4779-8d6e-12f28b47f203 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:04.374286 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6a50b-536c-4f1f-951a-63de640e2a90 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:04.412267 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0ffeb-ffcb-46aa-aa51-0c361219e080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:04.412882 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0ffeb-ffcb-46aa-aa51-0c361219e080 None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:04.413155 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0ffeb-ffcb-46aa-aa51-0c361219e080 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:04.413419 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0ffeb-ffcb-46aa-aa51-0c361219e080 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:04.414063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0ffeb-ffcb-46aa-aa51-0c361219e080 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:04.415682 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0ffeb-ffcb-46aa-aa51-0c361219e080 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:04.434545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 286/563] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Oct 12 14:27:04 2022] POST /identity/v3/services => generated 240 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:27:04.435717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0ffeb-ffcb-46aa-aa51-0c361219e080 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:04.514999 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dbedf-789d-4779-8d6e-12f28b47f203 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:04.624052 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94e2e1-8f82-4d1b-8b59-53e71558e888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:04.624052 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94e2e1-8f82-4d1b-8b59-53e71558e888 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:04.624052 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94e2e1-8f82-4d1b-8b59-53e71558e888 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:04.624052 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94e2e1-8f82-4d1b-8b59-53e71558e888 None admin] PATH_INFO: `/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/c8381a7c961047c69de894cf5b0055b6/roles/4b1f48ab911e4fadbc70164ae43d508b` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:04.627383 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94e2e1-8f82-4d1b-8b59-53e71558e888 None admin] RBAC: Authorizing `identity:create_grant(project_id=720b11237d2e48128c171be45fcd0ad0, user_id=c8381a7c961047c69de894cf5b0055b6, role_id=4b1f48ab911e4fadbc70164ae43d508b)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:04.629450 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94e2e1-8f82-4d1b-8b59-53e71558e888 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:04.646726 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:04.653761 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0ffeb-ffcb-46aa-aa51-0c361219e080 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:04.654654 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 278/564] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:27:04 2022] PUT /identity/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/c8381a7c961047c69de894cf5b0055b6/roles/4b1f48ab911e4fadbc70164ae43d508b => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:27:04.668793 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:04.701271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76869e7e-df26-405c-9b7c-826a8744e23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:04.701898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76869e7e-df26-405c-9b7c-826a8744e23c None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:04.702173 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76869e7e-df26-405c-9b7c-826a8744e23c None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:04.702439 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76869e7e-df26-405c-9b7c-826a8744e23c None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:04.703106 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76869e7e-df26-405c-9b7c-826a8744e23c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:04.703432 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76869e7e-df26-405c-9b7c-826a8744e23c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:04.704815 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76869e7e-df26-405c-9b7c-826a8744e23c None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:04.715373 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 287/565] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:27:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:04.724641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94e2e1-8f82-4d1b-8b59-53e71558e888 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:04.865527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94e2e1-8f82-4d1b-8b59-53e71558e888 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:04.917881 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e568de1-802f-4911-a42a-1082def13045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:04.918527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e568de1-802f-4911-a42a-1082def13045 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:04.918798 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e568de1-802f-4911-a42a-1082def13045 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:04.919065 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e568de1-802f-4911-a42a-1082def13045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:04.930817 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e568de1-802f-4911-a42a-1082def13045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:04.932301 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e568de1-802f-4911-a42a-1082def13045 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:04.949037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-7e568de1-802f-4911-a42a-1082def13045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:04.949037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:04.949037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:04.949037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:04.949037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:04.949037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:04.949037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:04.949037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:04.949037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:04.949037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:04.949037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:04.949037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:04.949037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:04.949037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:04.949037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:04.949037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:04.950013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:04.950855 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:04.950855 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:04.950855 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:04.950855 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 279/566] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:04.958156 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76869e7e-df26-405c-9b7c-826a8744e23c None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:05.064714 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76869e7e-df26-405c-9b7c-826a8744e23c None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:05.106585 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b70b10-22f5-4a51-a24f-0bf4a002b249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:05.107259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b70b10-22f5-4a51-a24f-0bf4a002b249 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:05.107541 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b70b10-22f5-4a51-a24f-0bf4a002b249 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:05.107807 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b70b10-22f5-4a51-a24f-0bf4a002b249 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:05.108462 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b70b10-22f5-4a51-a24f-0bf4a002b249 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:05.108762 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b70b10-22f5-4a51-a24f-0bf4a002b249 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:05.110170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b70b10-22f5-4a51-a24f-0bf4a002b249 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:05.123875 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 288/567] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:05.130289 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e568de1-802f-4911-a42a-1082def13045 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:05.204617 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:05.268991 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e568de1-802f-4911-a42a-1082def13045 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:05.310653 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75422cb6-c6b4-4840-9112-78ae567ebbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:05.311324 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75422cb6-c6b4-4840-9112-78ae567ebbc5 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:05.311604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75422cb6-c6b4-4840-9112-78ae567ebbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:05.311880 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75422cb6-c6b4-4840-9112-78ae567ebbc5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:05.322970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75422cb6-c6b4-4840-9112-78ae567ebbc5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:05.324453 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75422cb6-c6b4-4840-9112-78ae567ebbc5 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:05.339328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-75422cb6-c6b4-4840-9112-78ae567ebbc5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 14:27:05.339328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:05.339328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:05.339328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:05.339328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:05.339328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:05.339328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:05.339328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:05.339328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:05.339328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:05.339328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:05.339328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:05.339328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:27:05.339328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:27:05.339328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:27:05.339328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:05.341446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:05.344099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:05.344099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:05.344099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:27:05.344099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:27:05.344099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:27:05.344099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:27:05.344099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:27:05.344099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:27:05.344099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 14:27:05.344099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:05.344099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 280/568] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Oct 12 14:27:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:05.351244 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b70b10-22f5-4a51-a24f-0bf4a002b249 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:05.400630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:05.492014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b70b10-22f5-4a51-a24f-0bf4a002b249 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:05.537250 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96523212-ed7d-487f-9c7c-3ab9526b005b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:05.537871 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96523212-ed7d-487f-9c7c-3ab9526b005b None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:05.538144 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96523212-ed7d-487f-9c7c-3ab9526b005b None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:05.538414 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96523212-ed7d-487f-9c7c-3ab9526b005b None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:05.539160 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96523212-ed7d-487f-9c7c-3ab9526b005b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:05.539466 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96523212-ed7d-487f-9c7c-3ab9526b005b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:05.540897 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96523212-ed7d-487f-9c7c-3ab9526b005b None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:05.575059 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 289/569] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Oct 12 14:27:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:05.584696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75422cb6-c6b4-4840-9112-78ae567ebbc5 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:05.738929 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75422cb6-c6b4-4840-9112-78ae567ebbc5 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:05.755645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:05.789452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5843f-ea1b-4295-a299-033de327fc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:05.790081 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5843f-ea1b-4295-a299-033de327fc8c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:05.790355 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5843f-ea1b-4295-a299-033de327fc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:05.790623 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5843f-ea1b-4295-a299-033de327fc8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:05.887276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5843f-ea1b-4295-a299-033de327fc8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:05.888746 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5843f-ea1b-4295-a299-033de327fc8c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:06.024674 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 281/570] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:27:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 14:27:06.032318 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96523212-ed7d-487f-9c7c-3ab9526b005b None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:06.138731 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6add4-6b9a-4e32-ade2-e6cbd3e3cc90 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:06.139022 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6add4-6b9a-4e32-ade2-e6cbd3e3cc90 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:06.139316 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6add4-6b9a-4e32-ade2-e6cbd3e3cc90 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:06.140781 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 282/571] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:27:06.148232 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb2696-d41b-49e7-b378-1df881dcc775 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:06.148515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb2696-d41b-49e7-b378-1df881dcc775 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:06.148792 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb2696-d41b-49e7-b378-1df881dcc775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:06.179509 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:06.181958 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-7bfb2696-d41b-49e7-b378-1df881dcc775 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:06.183164 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7bfb2696-d41b-49e7-b378-1df881dcc775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:06.189128 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:06.202451 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96523212-ed7d-487f-9c7c-3ab9526b005b None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:06.263885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37597369-2424-490a-9d35-fea547b4a12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:06.264525 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37597369-2424-490a-9d35-fea547b4a12d None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:06.264798 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37597369-2424-490a-9d35-fea547b4a12d None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:06.265065 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37597369-2424-490a-9d35-fea547b4a12d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:06.285885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37597369-2424-490a-9d35-fea547b4a12d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:06.287292 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37597369-2424-490a-9d35-fea547b4a12d None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:06.307664 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 283/572] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:06.312788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-37597369-2424-490a-9d35-fea547b4a12d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:06.312788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:06.312788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:06.312788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:06.312788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:06.312788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:06.312788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:06.312788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:06.312788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:06.312788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:06.312788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:06.312788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:06.312788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:06.312788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:06.312788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:06.312788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:06.313746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:06.314629 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:06.314629 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:06.314629 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:06.314629 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 290/573] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:06.319604 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb2696-d41b-49e7-b378-1df881dcc775 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:06.427404 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb2696-d41b-49e7-b378-1df881dcc775 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:06.466898 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015251dd-b044-4601-9cb6-4098b20ab284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:06.467556 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015251dd-b044-4601-9cb6-4098b20ab284 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:06.467829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015251dd-b044-4601-9cb6-4098b20ab284 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:06.468098 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015251dd-b044-4601-9cb6-4098b20ab284 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:06.468757 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015251dd-b044-4601-9cb6-4098b20ab284 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:06.469056 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015251dd-b044-4601-9cb6-4098b20ab284 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:06.470441 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015251dd-b044-4601-9cb6-4098b20ab284 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:06.480717 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:06.490866 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 284/574] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:06.496471 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37597369-2424-490a-9d35-fea547b4a12d None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:06.594429 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895529c-9001-4fcf-9c21-4e7d7ccac351 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:06.594996 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895529c-9001-4fcf-9c21-4e7d7ccac351 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:06.595526 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895529c-9001-4fcf-9c21-4e7d7ccac351 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:06.597975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 285/575] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:06.603070 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daeb3d-de96-418b-b3d7-3410653193eb None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:06.603624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daeb3d-de96-418b-b3d7-3410653193eb None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:06.604094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daeb3d-de96-418b-b3d7-3410653193eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:06.625843 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-16daeb3d-de96-418b-b3d7-3410653193eb None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:06.627382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-16daeb3d-de96-418b-b3d7-3410653193eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:06.639734 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37597369-2424-490a-9d35-fea547b4a12d None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:06.689137 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922c8c2-3121-47cc-9532-68696ff13698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:06.691717 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922c8c2-3121-47cc-9532-68696ff13698 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:06.692239 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922c8c2-3121-47cc-9532-68696ff13698 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:06.692765 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922c8c2-3121-47cc-9532-68696ff13698 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:06.708286 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922c8c2-3121-47cc-9532-68696ff13698 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:06.710058 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922c8c2-3121-47cc-9532-68696ff13698 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:06.715484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:06.730630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-0922c8c2-3121-47cc-9532-68696ff13698 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:06.730630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:06.730630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:06.730630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:06.730630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:06.730630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:06.730630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:06.730630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:06.730630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:06.730630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:06.730630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:06.730630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:06.730630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:27:06.730630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:27:06.730630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:06.731616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:06.732491 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:27:06.732491 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:27:06.732491 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:27:06.732491 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:27:06.732491 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:27:06.732491 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:27:06.732491 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:06.732491 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:06.735230 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 291/576] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:27:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:06.739872 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922c8c2-3121-47cc-9532-68696ff13698 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:06.754550 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 286/577] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 14:27:06.783282 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16daeb3d-de96-418b-b3d7-3410653193eb None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:06.939698 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16daeb3d-de96-418b-b3d7-3410653193eb None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:06.957547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922c8c2-3121-47cc-9532-68696ff13698 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:06.991516 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:07.002052 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df85935-a9b4-43ff-ae5a-79b3c6468d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:07.002694 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df85935-a9b4-43ff-ae5a-79b3c6468d09 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:07.003065 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df85935-a9b4-43ff-ae5a-79b3c6468d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:07.003361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df85935-a9b4-43ff-ae5a-79b3c6468d09 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:07.003949 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df85935-a9b4-43ff-ae5a-79b3c6468d09 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:07.005346 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df85935-a9b4-43ff-ae5a-79b3c6468d09 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-5df85935-a9b4-43ff-ae5a-79b3c6468d09 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:07.020622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:07.022035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:07.022035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:07.022035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:07.022035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:07.022035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:07.022035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:07.022035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:07.022035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:07.022035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:07.022035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:07.022035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:07.022035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:07.022035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:07.022035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:07.022035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:07.022035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:07.023548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:07.023548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:07.023548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:07.023548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:07.023548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:07.023548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:07.023548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:07.023548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:07.023548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:07.023548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:07.023548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:07.023548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:07.023548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 14:27:07.023548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:07.023548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 287/578] 10.222.0.16 () {56 vars in 1129 bytes} [Wed Oct 12 14:27:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:07.029755 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df85935-a9b4-43ff-ae5a-79b3c6468d09 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:07.035324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66fc0f-b511-4c0b-9ff3-da8a7af43216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:07.036540 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66fc0f-b511-4c0b-9ff3-da8a7af43216 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:07.037033 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66fc0f-b511-4c0b-9ff3-da8a7af43216 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:07.037507 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66fc0f-b511-4c0b-9ff3-da8a7af43216 None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:07.038401 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66fc0f-b511-4c0b-9ff3-da8a7af43216 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:07.038910 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66fc0f-b511-4c0b-9ff3-da8a7af43216 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:07.040678 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66fc0f-b511-4c0b-9ff3-da8a7af43216 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:07.064602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 292/579] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:27:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:07.070629 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66fc0f-b511-4c0b-9ff3-da8a7af43216 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:07.148666 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:07.174635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df85935-a9b4-43ff-ae5a-79b3c6468d09 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:07.208902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66fc0f-b511-4c0b-9ff3-da8a7af43216 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:07.224162 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68adef-698c-4a9a-ad32-de3515ebebe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:07.224735 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68adef-698c-4a9a-ad32-de3515ebebe9 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:07.225013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68adef-698c-4a9a-ad32-de3515ebebe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:07.225279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68adef-698c-4a9a-ad32-de3515ebebe9 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:07.225908 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68adef-698c-4a9a-ad32-de3515ebebe9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:07.226202 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68adef-698c-4a9a-ad32-de3515ebebe9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:07.227611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68adef-698c-4a9a-ad32-de3515ebebe9 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:07.235305 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 288/580] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Oct 12 14:27:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:07.236438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68adef-698c-4a9a-ad32-de3515ebebe9 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:07.258315 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c92c4-4e7e-4f3e-bf03-4b0172074b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:07.259418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c92c4-4e7e-4f3e-bf03-4b0172074b53 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:07.259935 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c92c4-4e7e-4f3e-bf03-4b0172074b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:07.260421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c92c4-4e7e-4f3e-bf03-4b0172074b53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:07.261391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c92c4-4e7e-4f3e-bf03-4b0172074b53 None admin] RBAC: Adding query filter params (role.id=4b1f48ab911e4fadbc70164ae43d508b, scope.project.id=720b11237d2e48128c171be45fcd0ad0, user.id=d0aeed5f32a1477dbd9d3e934669705b) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:07.261946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c92c4-4e7e-4f3e-bf03-4b0172074b53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:07.263721 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c92c4-4e7e-4f3e-bf03-4b0172074b53 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:07.271559 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 293/581] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:27:07 2022] GET /identity/v3/role_assignments?user.id=d0aeed5f32a1477dbd9d3e934669705b&scope.project.id=720b11237d2e48128c171be45fcd0ad0&role.id=4b1f48ab911e4fadbc70164ae43d508b => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:07.273078 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c92c4-4e7e-4f3e-bf03-4b0172074b53 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:07.321577 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:07.373600 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68adef-698c-4a9a-ad32-de3515ebebe9 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:07.395833 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c92c4-4e7e-4f3e-bf03-4b0172074b53 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:07.416550 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404d975-070e-4ed2-acf1-5830d0c3365e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:07.417197 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404d975-070e-4ed2-acf1-5830d0c3365e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:07.417804 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404d975-070e-4ed2-acf1-5830d0c3365e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:07.417804 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404d975-070e-4ed2-acf1-5830d0c3365e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:07.422060 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:07.430245 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404d975-070e-4ed2-acf1-5830d0c3365e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:27:07.435833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-d404d975-070e-4ed2-acf1-5830d0c3365e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:27:07.435833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:07.435833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:07.435833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:07.435833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:07.435833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:07.435833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:07.435833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:07.435833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:07.435833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:07.435833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:07.435833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:07.435833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:27:07.435833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:27:07.435833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:27:07.435833 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:27:07.436824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:27:07.436824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:27:07.436824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:27:07.436824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:27:07.436824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:07.436824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:07.436824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:07.436824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:07.436824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:07.436824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:07.436824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:07.436824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:07.436824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:07.436824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:07.436824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:07.436824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:07.443431 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:07.443431 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:07.443431 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:07.443431 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:07.443431 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:27:07.443431 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:27:07.443431 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:27:07.443431 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:27:07.443431 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:27:07.443431 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:27:07.443431 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:27:07.443431 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:07.444093 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9638a47-e00c-4beb-89fb-f7a20cd750fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:07.444093 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9638a47-e00c-4beb-89fb-f7a20cd750fe None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:07.444093 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9638a47-e00c-4beb-89fb-f7a20cd750fe None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:07.444093 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9638a47-e00c-4beb-89fb-f7a20cd750fe None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:07.444374 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 289/582] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Oct 12 14:27:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:07.444447 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9638a47-e00c-4beb-89fb-f7a20cd750fe None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:07.444447 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9638a47-e00c-4beb-89fb-f7a20cd750fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:07.444581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e8057-7c27-4158-84d3-e62266c04e6b None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:07.444581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e8057-7c27-4158-84d3-e62266c04e6b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:07.444581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e8057-7c27-4158-84d3-e62266c04e6b None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:07.444770 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9638a47-e00c-4beb-89fb-f7a20cd750fe None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:07.445712 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 290/583] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:07.446894 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e907fd8-a55c-4b07-a510-2d7241ee81fb None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:07.447181 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e907fd8-a55c-4b07-a510-2d7241ee81fb None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:07.447453 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e907fd8-a55c-4b07-a510-2d7241ee81fb None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:07.448924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 291/584] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:07.449483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e907fd8-a55c-4b07-a510-2d7241ee81fb None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:07.456991 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 294/585] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Oct 12 14:27:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:07.456991 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb1a75-5f75-45c9-8072-4a03559a68b7 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:07.457306 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb1a75-5f75-45c9-8072-4a03559a68b7 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:07.457514 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb1a75-5f75-45c9-8072-4a03559a68b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:07.476907 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-1fdb1a75-5f75-45c9-8072-4a03559a68b7 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:07.477532 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-1fdb1a75-5f75-45c9-8072-4a03559a68b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:07.554076 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 295/586] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:07.554076 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a79ba9-997d-44dc-927b-820af1a28c7b None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:07.554076 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a79ba9-997d-44dc-927b-820af1a28c7b None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:07.554076 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a79ba9-997d-44dc-927b-820af1a28c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:07.570666 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-e9a79ba9-997d-44dc-927b-820af1a28c7b None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:07.572059 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-e9a79ba9-997d-44dc-927b-820af1a28c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:07.580305 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e907fd8-a55c-4b07-a510-2d7241ee81fb None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:07.605735 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:07.628742 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be459e65-0238-4bc0-b206-8f37b7fed9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:07.630137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be459e65-0238-4bc0-b206-8f37b7fed9af None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:07.630565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be459e65-0238-4bc0-b206-8f37b7fed9af None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:07.630993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be459e65-0238-4bc0-b206-8f37b7fed9af None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:07.631839 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be459e65-0238-4bc0-b206-8f37b7fed9af None admin] RBAC: Adding query filter params (name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:07.632403 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be459e65-0238-4bc0-b206-8f37b7fed9af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:07.634419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be459e65-0238-4bc0-b206-8f37b7fed9af None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:07.646841 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 292/587] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Oct 12 14:27:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:07.648089 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be459e65-0238-4bc0-b206-8f37b7fed9af None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:07.663487 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 296/588] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:07.664448 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a79ba9-997d-44dc-927b-820af1a28c7b None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:07.714704 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:07.767365 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be459e65-0238-4bc0-b206-8f37b7fed9af None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:07.767678 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a79ba9-997d-44dc-927b-820af1a28c7b None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:07.813793 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81d768-eb6d-405c-a41f-c4c28a226b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:07.815234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81d768-eb6d-405c-a41f-c4c28a226b18 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:07.816194 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5291b83-8e9f-47ed-ae66-771d058468ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:07.817063 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5291b83-8e9f-47ed-ae66-771d058468ab None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:07.817495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81d768-eb6d-405c-a41f-c4c28a226b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:07.817921 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5291b83-8e9f-47ed-ae66-771d058468ab None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:07.818342 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81d768-eb6d-405c-a41f-c4c28a226b18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:07.819369 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5291b83-8e9f-47ed-ae66-771d058468ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:07.820283 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5291b83-8e9f-47ed-ae66-771d058468ab None admin] RBAC: Adding query filter params (interface=public, service_id=ceb7e2c21ba84cffa5cb26fac0138cbb, region_id=RegionOne) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:07.820975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5291b83-8e9f-47ed-ae66-771d058468ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:07.823620 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5291b83-8e9f-47ed-ae66-771d058468ab None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:07.838614 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81d768-eb6d-405c-a41f-c4c28a226b18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:07.841353 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 293/589] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Oct 12 14:27:07 2022] GET /identity/v3/endpoints?service_id=ceb7e2c21ba84cffa5cb26fac0138cbb&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:07.842460 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81d768-eb6d-405c-a41f-c4c28a226b18 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:07.847673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5291b83-8e9f-47ed-ae66-771d058468ab None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:07.875824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-9c81d768-eb6d-405c-a41f-c4c28a226b18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:07.875824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:07.875824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:07.875824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:07.875824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:07.875824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:07.875824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:07.875824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:07.875824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:07.875824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:07.875824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:07.875824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:07.875824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:07.875824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:07.875824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:07.875824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:07.877064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:07.878678 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:07.878678 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:07.878678 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:07.883040 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 297/590] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:07.886766 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81d768-eb6d-405c-a41f-c4c28a226b18 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:07.929456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:07.997693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:08.000435 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81d768-eb6d-405c-a41f-c4c28a226b18 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:08.001821 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5291b83-8e9f-47ed-ae66-771d058468ab None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:08.038464 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc3769-1fa4-4a2d-9427-9b146ba317d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:08.039350 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc3769-1fa4-4a2d-9427-9b146ba317d2 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:08.039632 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc3769-1fa4-4a2d-9427-9b146ba317d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:08.039898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc3769-1fa4-4a2d-9427-9b146ba317d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:08.040542 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc3769-1fa4-4a2d-9427-9b146ba317d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:08.040842 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc3769-1fa4-4a2d-9427-9b146ba317d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:08.040948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1208b-9f0b-4f9b-ba1b-1e6f1e0073de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:08.041550 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1208b-9f0b-4f9b-ba1b-1e6f1e0073de None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:08.041823 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1208b-9f0b-4f9b-ba1b-1e6f1e0073de None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:08.042088 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1208b-9f0b-4f9b-ba1b-1e6f1e0073de None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:08.042190 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc3769-1fa4-4a2d-9427-9b146ba317d2 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:08.042684 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1208b-9f0b-4f9b-ba1b-1e6f1e0073de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:08.044054 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1208b-9f0b-4f9b-ba1b-1e6f1e0073de None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:08.049409 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 294/591] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Oct 12 14:27:07 2022] GET /identity/v3/services => generated 772 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:08.055189 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 298/592] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:08.056066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1208b-9f0b-4f9b-ba1b-1e6f1e0073de None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:08.056186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc3769-1fa4-4a2d-9427-9b146ba317d2 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:08.172013 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc3769-1fa4-4a2d-9427-9b146ba317d2 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:08.192988 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1208b-9f0b-4f9b-ba1b-1e6f1e0073de None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:08.207135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:08.217832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bac90-92ae-48f1-a702-26ef03228adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:08.219166 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bac90-92ae-48f1-a702-26ef03228adc None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:08.220162 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bac90-92ae-48f1-a702-26ef03228adc None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:08.220643 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bac90-92ae-48f1-a702-26ef03228adc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:08.237614 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bac90-92ae-48f1-a702-26ef03228adc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:27:08.240616 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1911870-1be2-40ea-897b-724fdfbd4857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:08.241263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1911870-1be2-40ea-897b-724fdfbd4857 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:08.241688 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1911870-1be2-40ea-897b-724fdfbd4857 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:08.241969 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1911870-1be2-40ea-897b-724fdfbd4857 None admin] PATH_INFO: `/v3/services/network` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:08.242561 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1911870-1be2-40ea-897b-724fdfbd4857 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:08.244038 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1911870-1be2-40ea-897b-724fdfbd4857 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:08.244510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-110bac90-92ae-48f1-a702-26ef03228adc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:27:08.244510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:08.244510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:08.244510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:08.244510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:08.244510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:08.244510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:08.244510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:08.244510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:08.244510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:08.244510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:08.244510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:08.244510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:27:08.244510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:27:08.244510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:27:08.244510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:27:08.245423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:27:08.245423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:27:08.245423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:27:08.245423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:27:08.245423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:08.245423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:08.245423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:08.245423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:08.245423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:08.245423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:08.245423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:08.245423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:08.245423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:08.245423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:08.245423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:08.245423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:08.246269 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:08.246269 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:08.246269 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:08.246269 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:08.246269 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:27:08.246269 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:27:08.246269 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:27:08.246269 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:27:08.246269 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:27:08.246269 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:27:08.246269 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:27:08.246269 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:08.249328 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 299/593] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:27:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:08.250285 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bac90-92ae-48f1-a702-26ef03228adc None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-b1911870-1be2-40ea-897b-724fdfbd4857 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:08.260611 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:08.261876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:08.261876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:08.261876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:08.261876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:08.261876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:08.261876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:08.261876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:08.261876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:08.261876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:08.261876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:08.261876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:08.261876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:08.261876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:08.261876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:08.261876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:08.261876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:08.262764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:08.262764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:08.262764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:08.262764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:08.262764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:08.262764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:08.262764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:08.262764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:08.262764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:08.262764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:08.262764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:08.262764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:08.262764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 14:27:08.262764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:08.263515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 295/594] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:27:08 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:08.264230 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1911870-1be2-40ea-897b-724fdfbd4857 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:08.283817 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:08.380353 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1911870-1be2-40ea-897b-724fdfbd4857 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:08.393260 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bac90-92ae-48f1-a702-26ef03228adc None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:08.422942 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd41e3-99bd-43a8-a8e9-07736c01d385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:08.424135 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd41e3-99bd-43a8-a8e9-07736c01d385 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:08.424536 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd41e3-99bd-43a8-a8e9-07736c01d385 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:08.425438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd41e3-99bd-43a8-a8e9-07736c01d385 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:08.426600 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd41e3-99bd-43a8-a8e9-07736c01d385 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:08.427280 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd41e3-99bd-43a8-a8e9-07736c01d385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:08.429437 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd41e3-99bd-43a8-a8e9-07736c01d385 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:08.433293 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a4be2-a499-41ef-be18-79dc88351e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:08.434108 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a4be2-a499-41ef-be18-79dc88351e28 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:08.434536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a4be2-a499-41ef-be18-79dc88351e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:08.434948 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a4be2-a499-41ef-be18-79dc88351e28 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:08.443231 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 296/595] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:27:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:08.444692 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd41e3-99bd-43a8-a8e9-07736c01d385 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:08.446633 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a4be2-a499-41ef-be18-79dc88351e28 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:08.448268 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a4be2-a499-41ef-be18-79dc88351e28 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:08.462317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-920a4be2-a499-41ef-be18-79dc88351e28 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 14:27:08.462317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:08.462317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:08.462317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:08.462317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:08.462317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:08.462317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:08.462317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:08.462317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:08.462317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:08.462317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:08.462317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:08.462317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:27:08.462317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:27:08.462317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:27:08.462317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:08.463550 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:08.464675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:08.464675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:08.464675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:27:08.464675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:27:08.464675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:27:08.464675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:27:08.464675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:27:08.464675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:27:08.464675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 14:27:08.464675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:08.466805 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 300/596] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Oct 12 14:27:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:08.467762 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a4be2-a499-41ef-be18-79dc88351e28 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:08.493065 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:08.560000 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:08.589706 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a4be2-a499-41ef-be18-79dc88351e28 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:08.591620 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd41e3-99bd-43a8-a8e9-07736c01d385 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:08.625410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7151a-eaaf-41a0-8c55-28eec0c1107c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:08.626177 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7151a-eaaf-41a0-8c55-28eec0c1107c None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:08.626588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7151a-eaaf-41a0-8c55-28eec0c1107c None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:08.626947 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7151a-eaaf-41a0-8c55-28eec0c1107c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:08.638587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f353cd-2d8a-4c86-ae50-dcccd05f5aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:08.639131 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f353cd-2d8a-4c86-ae50-dcccd05f5aed None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:08.639372 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7151a-eaaf-41a0-8c55-28eec0c1107c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:08.639483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f353cd-2d8a-4c86-ae50-dcccd05f5aed None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:08.639685 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f353cd-2d8a-4c86-ae50-dcccd05f5aed None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:08.640304 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f353cd-2d8a-4c86-ae50-dcccd05f5aed None admin] RBAC: Adding query filter params (name=network) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:08.640644 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f353cd-2d8a-4c86-ae50-dcccd05f5aed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:08.641280 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7151a-eaaf-41a0-8c55-28eec0c1107c None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:08.642186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f353cd-2d8a-4c86-ae50-dcccd05f5aed None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:08.654414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 297/597] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Oct 12 14:27:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:08.655743 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f353cd-2d8a-4c86-ae50-dcccd05f5aed None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:08.661485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-86c7151a-eaaf-41a0-8c55-28eec0c1107c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:08.661485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:08.661485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:08.661485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:08.661485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:08.661485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:08.661485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:08.661485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:08.661485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:08.661485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:08.661485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:08.661485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:08.661485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:08.661485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:08.661485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:08.661485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:08.662593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:08.663709 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:08.663709 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:08.663709 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:08.667167 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 301/598] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:08.667167 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7151a-eaaf-41a0-8c55-28eec0c1107c None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:08.778408 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:08.782620 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7151a-eaaf-41a0-8c55-28eec0c1107c None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:08.785019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f353cd-2d8a-4c86-ae50-dcccd05f5aed None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:08.813787 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:08.821044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679780a4-a37e-4a4d-a90c-aaf0bd8daee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:08.821765 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679780a4-a37e-4a4d-a90c-aaf0bd8daee9 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:08.822138 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679780a4-a37e-4a4d-a90c-aaf0bd8daee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:08.822722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679780a4-a37e-4a4d-a90c-aaf0bd8daee9 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:08.823257 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679780a4-a37e-4a4d-a90c-aaf0bd8daee9 None admin] RBAC: Adding query filter params (type=network) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:08.823703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679780a4-a37e-4a4d-a90c-aaf0bd8daee9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:08.825168 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679780a4-a37e-4a4d-a90c-aaf0bd8daee9 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:08.835416 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 302/599] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Oct 12 14:27:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:08.835416 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679780a4-a37e-4a4d-a90c-aaf0bd8daee9 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:08.836851 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1e1b8-b18a-46a7-b278-824ec19eb730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:08.839272 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1e1b8-b18a-46a7-b278-824ec19eb730 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:08.839616 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1e1b8-b18a-46a7-b278-824ec19eb730 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:08.839954 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1e1b8-b18a-46a7-b278-824ec19eb730 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:08.841421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1e1b8-b18a-46a7-b278-824ec19eb730 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:08.841421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1e1b8-b18a-46a7-b278-824ec19eb730 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:08.842565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1e1b8-b18a-46a7-b278-824ec19eb730 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:08.861792 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 298/600] 10.222.0.16 () {56 vars in 1149 bytes} [Wed Oct 12 14:27:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:08.862928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1e1b8-b18a-46a7-b278-824ec19eb730 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:08.973378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1e1b8-b18a-46a7-b278-824ec19eb730 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:08.978376 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679780a4-a37e-4a4d-a90c-aaf0bd8daee9 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:09.011048 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbe07f-505b-45c3-8116-88e32ab0ad79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:09.011848 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbe07f-505b-45c3-8116-88e32ab0ad79 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:09.012200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbe07f-505b-45c3-8116-88e32ab0ad79 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:09.012541 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbe07f-505b-45c3-8116-88e32ab0ad79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:09.013330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afbe07f-505b-45c3-8116-88e32ab0ad79 None admin] RBAC: Adding query filter params (interface=public, service_id=feeb8fb792f8472b8e6818cf82158d41, region_id=RegionOne) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:09.013705 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afbe07f-505b-45c3-8116-88e32ab0ad79 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:09.015245 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afbe07f-505b-45c3-8116-88e32ab0ad79 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:09.023075 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 299/601] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Oct 12 14:27:08 2022] GET /identity/v3/endpoints?service_id=feeb8fb792f8472b8e6818cf82158d41&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:09.023075 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbe07f-505b-45c3-8116-88e32ab0ad79 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:09.033519 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61867364-baec-4218-af8c-376bd5edd73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:09.034461 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61867364-baec-4218-af8c-376bd5edd73a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:09.035715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61867364-baec-4218-af8c-376bd5edd73a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:09.036193 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61867364-baec-4218-af8c-376bd5edd73a None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:09.037008 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61867364-baec-4218-af8c-376bd5edd73a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:09.037920 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61867364-baec-4218-af8c-376bd5edd73a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:09.039928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61867364-baec-4218-af8c-376bd5edd73a None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:09.052220 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:09.059180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 303/602] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:09.059180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61867364-baec-4218-af8c-376bd5edd73a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:09.132738 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:09.153786 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbe07f-505b-45c3-8116-88e32ab0ad79 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:09.182289 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61867364-baec-4218-af8c-376bd5edd73a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:09.200300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17246e0-2e43-4275-82cf-990d2820a804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:09.200909 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17246e0-2e43-4275-82cf-990d2820a804 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:09.201184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17246e0-2e43-4275-82cf-990d2820a804 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:09.201450 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17246e0-2e43-4275-82cf-990d2820a804 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:09.221620 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa96a39-38da-4bb5-9c3b-7727fa05be09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:09.223168 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa96a39-38da-4bb5-9c3b-7727fa05be09 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:09.223769 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa96a39-38da-4bb5-9c3b-7727fa05be09 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:09.224337 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa96a39-38da-4bb5-9c3b-7727fa05be09 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:09.225168 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa96a39-38da-4bb5-9c3b-7727fa05be09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:09.226893 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa96a39-38da-4bb5-9c3b-7727fa05be09 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:09.232771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 304/603] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Oct 12 14:27:09 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:09.233859 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa96a39-38da-4bb5-9c3b-7727fa05be09 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:09.334890 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17246e0-2e43-4275-82cf-990d2820a804 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:09.337623 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17246e0-2e43-4275-82cf-990d2820a804 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:09.383927 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:09.437561 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa96a39-38da-4bb5-9c3b-7727fa05be09 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:09.462007 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:09.483248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769a11e-b774-42e9-a7da-71a47a1c7e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:09.484940 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769a11e-b774-42e9-a7da-71a47a1c7e35 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:09.486595 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769a11e-b774-42e9-a7da-71a47a1c7e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:09.487340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769a11e-b774-42e9-a7da-71a47a1c7e35 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:09.499350 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769a11e-b774-42e9-a7da-71a47a1c7e35 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:09.501995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769a11e-b774-42e9-a7da-71a47a1c7e35 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:09.519781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-8769a11e-b774-42e9-a7da-71a47a1c7e35 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 12 14:27:09.519781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:09.519781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:09.519781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:09.519781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:09.519781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:09.519781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:09.519781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:09.519781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:09.519781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:09.519781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:09.519781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:09.519781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:27:09.519781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:27:09.519781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:27:09.519781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:09.520808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:09.522052 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:09.522052 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:09.522052 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:27:09.522052 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:27:09.522052 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:27:09.522052 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:27:09.522052 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:27:09.522052 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:27:09.522052 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 12 14:27:09.522052 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:09.524661 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 305/604] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:27:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:09.528278 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 300/605] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:27:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 14:27:09.531015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87204f-cf52-4a61-8787-0eece3ff2fb6 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:09.531328 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87204f-cf52-4a61-8787-0eece3ff2fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:09.531598 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87204f-cf52-4a61-8787-0eece3ff2fb6 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:09.532902 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 306/606] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:09.533813 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17246e0-2e43-4275-82cf-990d2820a804 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:09.535532 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87204f-cf52-4a61-8787-0eece3ff2fb6 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:09.650479 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17246e0-2e43-4275-82cf-990d2820a804 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:09.674698 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:09.678674 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87204f-cf52-4a61-8787-0eece3ff2fb6 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:09.697108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed646fa-781a-4d46-9fb2-01e2858f9913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:09.697952 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed646fa-781a-4d46-9fb2-01e2858f9913 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:09.698389 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed646fa-781a-4d46-9fb2-01e2858f9913 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:09.698796 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed646fa-781a-4d46-9fb2-01e2858f9913 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:09.700238 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed646fa-781a-4d46-9fb2-01e2858f9913 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:09.700687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed646fa-781a-4d46-9fb2-01e2858f9913 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:09.702782 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed646fa-781a-4d46-9fb2-01e2858f9913 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:09.714431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930b55a-a808-4384-a86b-814b3624341c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:09.715033 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930b55a-a808-4384-a86b-814b3624341c None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:09.715386 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930b55a-a808-4384-a86b-814b3624341c None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:09.715659 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930b55a-a808-4384-a86b-814b3624341c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:09.727383 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930b55a-a808-4384-a86b-814b3624341c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:09.727922 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 301/607] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Oct 12 14:27:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:09.728835 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930b55a-a808-4384-a86b-814b3624341c None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:09.729657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86623f97-7f16-4a58-9bb1-e54953e4f8e7 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:09.730090 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86623f97-7f16-4a58-9bb1-e54953e4f8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:09.730781 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86623f97-7f16-4a58-9bb1-e54953e4f8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:09.744181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-4930b55a-a808-4384-a86b-814b3624341c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:09.744181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:09.744181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:09.744181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:09.744181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:09.744181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:09.744181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:09.744181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:09.744181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:09.744181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:09.744181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:09.744181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:09.744181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:09.744181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:09.744181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:09.744181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:09.745395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:09.746591 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:09.746591 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:09.746591 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:09.749881 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 307/608] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:09.750731 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930b55a-a808-4384-a86b-814b3624341c None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:09.756197 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-86623f97-7f16-4a58-9bb1-e54953e4f8e7 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:09.757380 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-86623f97-7f16-4a58-9bb1-e54953e4f8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:09.797044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:09.833707 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 302/609] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:09.834904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86623f97-7f16-4a58-9bb1-e54953e4f8e7 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:09.922847 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930b55a-a808-4384-a86b-814b3624341c None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:09.964548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86623f97-7f16-4a58-9bb1-e54953e4f8e7 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:09.974794 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:10.002713 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65d57c-fd3f-4eef-af2c-3740a192ac30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:10.003359 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65d57c-fd3f-4eef-af2c-3740a192ac30 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:10.003650 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65d57c-fd3f-4eef-af2c-3740a192ac30 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:10.003919 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65d57c-fd3f-4eef-af2c-3740a192ac30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:10.006099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8bc4f-d2c6-4d8e-acb4-d0fb74c07eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:10.006705 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8bc4f-d2c6-4d8e-acb4-d0fb74c07eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:10.006978 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8bc4f-d2c6-4d8e-acb4-d0fb74c07eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:10.007277 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8bc4f-d2c6-4d8e-acb4-d0fb74c07eb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:10.007926 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8bc4f-d2c6-4d8e-acb4-d0fb74c07eb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:10.008225 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8bc4f-d2c6-4d8e-acb4-d0fb74c07eb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:10.011297 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8bc4f-d2c6-4d8e-acb4-d0fb74c07eb2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:10.033567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 303/610] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:10.044353 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8bc4f-d2c6-4d8e-acb4-d0fb74c07eb2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:10.093956 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65d57c-fd3f-4eef-af2c-3740a192ac30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:10.095457 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65d57c-fd3f-4eef-af2c-3740a192ac30 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:10.155814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:10.216590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8bc4f-d2c6-4d8e-acb4-d0fb74c07eb2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:10.236701 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 308/611] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:27:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 14:27:10.237973 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65d57c-fd3f-4eef-af2c-3740a192ac30 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:10.261557 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1ca0b-c12c-4b59-99bf-0d3428639848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:10.262110 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1ca0b-c12c-4b59-99bf-0d3428639848 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:10.262383 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1ca0b-c12c-4b59-99bf-0d3428639848 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:10.262649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1ca0b-c12c-4b59-99bf-0d3428639848 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:10.270769 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1ca0b-c12c-4b59-99bf-0d3428639848 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:10.273484 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1ca0b-c12c-4b59-99bf-0d3428639848 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:10.286505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-28d1ca0b-c12c-4b59-99bf-0d3428639848 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:10.286505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:10.286505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:10.286505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:10.286505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:10.286505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:10.286505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:10.286505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:10.286505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:10.286505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:10.286505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:10.286505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:10.286505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:27:10.286505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:27:10.286505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:10.287454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:10.288747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:27:10.288747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:27:10.288747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:27:10.288747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:27:10.288747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:27:10.288747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:27:10.288747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:10.288747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:10.288747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 304/612] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:27:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:10.288747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1ca0b-c12c-4b59-99bf-0d3428639848 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:10.324363 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:10.352321 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65d57c-fd3f-4eef-af2c-3740a192ac30 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:10.386868 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:10.392924 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eec14-5e54-47df-ac32-9bb233eaf177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:10.393513 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eec14-5e54-47df-ac32-9bb233eaf177 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:10.394287 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eec14-5e54-47df-ac32-9bb233eaf177 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:10.394463 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eec14-5e54-47df-ac32-9bb233eaf177 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:10.395057 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95eec14-5e54-47df-ac32-9bb233eaf177 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:10.396520 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95eec14-5e54-47df-ac32-9bb233eaf177 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:10.404783 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1ca0b-c12c-4b59-99bf-0d3428639848 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:10.407776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-f95eec14-5e54-47df-ac32-9bb233eaf177 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 14:27:10.407776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:10.407776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:10.407776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:10.407776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:10.407776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:10.407776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:10.407776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:10.407776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 14:27:10.407776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:10.407776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:10.407776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:10.407776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:10.407776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:10.407776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:10.407776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:10.408683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:10.410538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:10.410538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:10.410538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:10.410538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:10.410538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:10.410538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:10.410538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:10.410538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:10.410538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:10.410538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:10.410538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:10.410538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:10.410538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 14:27:10.410538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:10.410538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 309/613] 10.222.0.16 () {56 vars in 1132 bytes} [Wed Oct 12 14:27:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:10.411771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eec14-5e54-47df-ac32-9bb233eaf177 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:10.449736 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163db5d-3bd6-4db6-a761-8dcb961e78ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:10.450371 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163db5d-3bd6-4db6-a761-8dcb961e78ba None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:10.450648 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163db5d-3bd6-4db6-a761-8dcb961e78ba None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:10.450921 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163db5d-3bd6-4db6-a761-8dcb961e78ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:10.463122 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163db5d-3bd6-4db6-a761-8dcb961e78ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:10.464584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163db5d-3bd6-4db6-a761-8dcb961e78ba None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:10.479186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-4163db5d-3bd6-4db6-a761-8dcb961e78ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:10.479186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:10.479186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:10.479186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:10.479186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:10.479186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:10.479186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:10.479186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:10.479186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:10.479186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:10.479186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:10.479186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:10.479186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:10.479186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:10.479186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:10.479186 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:10.480203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:10.481068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:10.481068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:10.481068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:10.484923 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 305/614] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:10.485975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc2980-da3e-4bfe-a49a-ecc8d21dafc4 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:10.486258 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc2980-da3e-4bfe-a49a-ecc8d21dafc4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:10.486532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc2980-da3e-4bfe-a49a-ecc8d21dafc4 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:10.487858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 306/615] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:10.488552 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc2980-da3e-4bfe-a49a-ecc8d21dafc4 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:10.520258 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eec14-5e54-47df-ac32-9bb233eaf177 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:10.554993 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a64a2-cde8-4bb1-8b41-a4d752e4d46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:10.555621 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a64a2-cde8-4bb1-8b41-a4d752e4d46a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:10.555897 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a64a2-cde8-4bb1-8b41-a4d752e4d46a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:10.556167 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a64a2-cde8-4bb1-8b41-a4d752e4d46a None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:10.556850 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a64a2-cde8-4bb1-8b41-a4d752e4d46a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:10.557146 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a64a2-cde8-4bb1-8b41-a4d752e4d46a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:10.558649 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a64a2-cde8-4bb1-8b41-a4d752e4d46a None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:10.571462 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 310/616] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:27:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:10.572068 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a64a2-cde8-4bb1-8b41-a4d752e4d46a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:10.584237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:10.605807 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc2980-da3e-4bfe-a49a-ecc8d21dafc4 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:10.635131 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:10.642420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325a688-5e61-4fa4-a31a-4ca4759edfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:10.643237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325a688-5e61-4fa4-a31a-4ca4759edfaf None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:10.643571 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325a688-5e61-4fa4-a31a-4ca4759edfaf None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:10.644555 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325a688-5e61-4fa4-a31a-4ca4759edfaf None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:10.645353 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325a688-5e61-4fa4-a31a-4ca4759edfaf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:10.645700 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325a688-5e61-4fa4-a31a-4ca4759edfaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:10.647841 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325a688-5e61-4fa4-a31a-4ca4759edfaf None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:10.652400 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 307/617] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Oct 12 14:27:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:10.657828 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730cd48-1bb1-46a4-b7eb-77a47b0ab16f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:10.658108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730cd48-1bb1-46a4-b7eb-77a47b0ab16f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:10.658388 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730cd48-1bb1-46a4-b7eb-77a47b0ab16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:10.676628 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-0730cd48-1bb1-46a4-b7eb-77a47b0ab16f None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:10.677839 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0730cd48-1bb1-46a4-b7eb-77a47b0ab16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:10.682766 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a64a2-cde8-4bb1-8b41-a4d752e4d46a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:10.723947 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155ef80-06b4-429a-9075-62b0f0f004a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:10.724646 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155ef80-06b4-429a-9075-62b0f0f004a6 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:10.724991 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155ef80-06b4-429a-9075-62b0f0f004a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:10.725329 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155ef80-06b4-429a-9075-62b0f0f004a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:10.726035 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155ef80-06b4-429a-9075-62b0f0f004a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:10.726402 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155ef80-06b4-429a-9075-62b0f0f004a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:10.727861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155ef80-06b4-429a-9075-62b0f0f004a6 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:10.740049 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 311/618] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:10.740933 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155ef80-06b4-429a-9075-62b0f0f004a6 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:10.752832 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 308/619] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:10.756421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730cd48-1bb1-46a4-b7eb-77a47b0ab16f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:10.849634 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155ef80-06b4-429a-9075-62b0f0f004a6 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:10.866622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:10.868983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:10.889707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15be9e7-d0a9-4ae1-b7ac-a974309d2065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:10.890660 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15be9e7-d0a9-4ae1-b7ac-a974309d2065 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:10.891031 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15be9e7-d0a9-4ae1-b7ac-a974309d2065 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:10.891436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15be9e7-d0a9-4ae1-b7ac-a974309d2065 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:10.892352 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15be9e7-d0a9-4ae1-b7ac-a974309d2065 None admin] RBAC: Adding query filter params (role.id=745dcd2d05e447538a1aaff54b38193c, scope.project.id=720b11237d2e48128c171be45fcd0ad0, user.id=15d7cb817428475e80cf697ac94c62b2) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:10.892882 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15be9e7-d0a9-4ae1-b7ac-a974309d2065 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:10.894610 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15be9e7-d0a9-4ae1-b7ac-a974309d2065 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:10.906072 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 312/620] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:27:10 2022] GET /identity/v3/role_assignments?user.id=15d7cb817428475e80cf697ac94c62b2&scope.project.id=720b11237d2e48128c171be45fcd0ad0&role.id=745dcd2d05e447538a1aaff54b38193c => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:10.906072 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15be9e7-d0a9-4ae1-b7ac-a974309d2065 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:10.906366 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730cd48-1bb1-46a4-b7eb-77a47b0ab16f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:10.982330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe2ce4-98ad-4260-bd68-c4392ca4b9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:10.982966 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe2ce4-98ad-4260-bd68-c4392ca4b9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:10.983325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe2ce4-98ad-4260-bd68-c4392ca4b9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:10.983596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe2ce4-98ad-4260-bd68-c4392ca4b9b8 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:10.984296 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe2ce4-98ad-4260-bd68-c4392ca4b9b8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:10.984597 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe2ce4-98ad-4260-bd68-c4392ca4b9b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:10.986050 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe2ce4-98ad-4260-bd68-c4392ca4b9b8 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:10.997250 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 309/621] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Oct 12 14:27:10 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:11.040293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe2ce4-98ad-4260-bd68-c4392ca4b9b8 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:11.092347 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15be9e7-d0a9-4ae1-b7ac-a974309d2065 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:11.134981 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c55d1-aef5-41d3-a0a5-6756dd824dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:11.135620 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c55d1-aef5-41d3-a0a5-6756dd824dce None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:11.135833 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c55d1-aef5-41d3-a0a5-6756dd824dce None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:11.136162 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c55d1-aef5-41d3-a0a5-6756dd824dce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:11.143502 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c55d1-aef5-41d3-a0a5-6756dd824dce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:11.145331 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c55d1-aef5-41d3-a0a5-6756dd824dce None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:11.158313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-d24c55d1-aef5-41d3-a0a5-6756dd824dce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:11.158313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:11.158313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:11.158313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:11.158313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:11.158313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:11.158313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:11.158313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:11.158313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:11.158313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:11.158313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:11.158313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:11.158313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:27:11.158313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:27:11.158313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:11.159304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:11.161039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:27:11.161039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:27:11.161039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:27:11.161039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:27:11.161039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:27:11.161039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:27:11.161039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:11.161039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:11.161039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 313/622] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:27:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:11.166313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c55d1-aef5-41d3-a0a5-6756dd824dce None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:11.215966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:11.256346 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe2ce4-98ad-4260-bd68-c4392ca4b9b8 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:11.307361 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c55d1-aef5-41d3-a0a5-6756dd824dce None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:11.336229 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca60173-344a-40f2-8ba6-21c08b071f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:11.336915 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca60173-344a-40f2-8ba6-21c08b071f8e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:11.337245 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca60173-344a-40f2-8ba6-21c08b071f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:11.337514 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca60173-344a-40f2-8ba6-21c08b071f8e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:11.338100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca60173-344a-40f2-8ba6-21c08b071f8e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:11.339489 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca60173-344a-40f2-8ba6-21c08b071f8e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-3ca60173-344a-40f2-8ba6-21c08b071f8e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:11.355657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:11.356696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:11.356696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:11.356696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:11.356696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:11.356696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:11.356696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:11.356696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:11.356696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:11.356696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:11.356696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:11.356696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:11.356696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:11.356696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:11.356696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:11.356696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:11.356696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:11.357529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:11.357529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:11.357529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:11.357529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:11.357529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:11.357529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:11.357529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:11.357529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:11.357529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:11.357529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:11.357529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:11.357529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:11.357529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 14:27:11.357529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:11.357529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 310/623] 10.222.0.16 () {56 vars in 1129 bytes} [Wed Oct 12 14:27:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:11.370330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca60173-344a-40f2-8ba6-21c08b071f8e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:11.374831 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:11.387108 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0100ebd-23a1-49a0-ae90-8f25c0421cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:11.387881 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0100ebd-23a1-49a0-ae90-8f25c0421cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:11.388268 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0100ebd-23a1-49a0-ae90-8f25c0421cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:11.388547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0100ebd-23a1-49a0-ae90-8f25c0421cc2 None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:11.389240 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0100ebd-23a1-49a0-ae90-8f25c0421cc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:11.389537 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0100ebd-23a1-49a0-ae90-8f25c0421cc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:11.391173 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0100ebd-23a1-49a0-ae90-8f25c0421cc2 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:11.416892 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 314/624] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:27:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:11.422917 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0100ebd-23a1-49a0-ae90-8f25c0421cc2 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:11.546058 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca60173-344a-40f2-8ba6-21c08b071f8e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:11.573163 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:11.578042 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0100ebd-23a1-49a0-ae90-8f25c0421cc2 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:11.617503 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635de2da-a355-45b6-b6eb-b88a04dc9dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:11.618109 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635de2da-a355-45b6-b6eb-b88a04dc9dfe None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:11.618384 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635de2da-a355-45b6-b6eb-b88a04dc9dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:11.618660 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635de2da-a355-45b6-b6eb-b88a04dc9dfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:11.619403 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635de2da-a355-45b6-b6eb-b88a04dc9dfe None admin] RBAC: Adding query filter params (role.id=4b1f48ab911e4fadbc70164ae43d508b, scope.project.id=720b11237d2e48128c171be45fcd0ad0, user.id=c8381a7c961047c69de894cf5b0055b6) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:11.619710 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635de2da-a355-45b6-b6eb-b88a04dc9dfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:11.621272 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635de2da-a355-45b6-b6eb-b88a04dc9dfe None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:11.627599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 315/625] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:27:11 2022] GET /identity/v3/role_assignments?user.id=c8381a7c961047c69de894cf5b0055b6&scope.project.id=720b11237d2e48128c171be45fcd0ad0&role.id=4b1f48ab911e4fadbc70164ae43d508b => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:11.657993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ac0f0-8cd9-485e-8b93-44f80a01c324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:11.658683 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ac0f0-8cd9-485e-8b93-44f80a01c324 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:11.658906 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ac0f0-8cd9-485e-8b93-44f80a01c324 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:11.659203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ac0f0-8cd9-485e-8b93-44f80a01c324 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:11.660424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ac0f0-8cd9-485e-8b93-44f80a01c324 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:11.660734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ac0f0-8cd9-485e-8b93-44f80a01c324 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:11.662333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ac0f0-8cd9-485e-8b93-44f80a01c324 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:11.668015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 311/626] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Oct 12 14:27:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:11.673735 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635de2da-a355-45b6-b6eb-b88a04dc9dfe None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:11.814122 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4b351-b7ef-4a7c-afb1-ed379f9e985d None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:11.814492 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4b351-b7ef-4a7c-afb1-ed379f9e985d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:11.814669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4b351-b7ef-4a7c-afb1-ed379f9e985d None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:11.818096 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 312/627] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:27:11.821461 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635de2da-a355-45b6-b6eb-b88a04dc9dfe None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:11.822455 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203799af-10c4-4ad9-ab03-0337fc7c4de8 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:11.822736 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203799af-10c4-4ad9-ab03-0337fc7c4de8 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:11.823010 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203799af-10c4-4ad9-ab03-0337fc7c4de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:11.859499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-203799af-10c4-4ad9-ab03-0337fc7c4de8 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:11.860250 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07704368-7dba-4bea-85b0-a54d41bd2fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:11.860731 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-203799af-10c4-4ad9-ab03-0337fc7c4de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:11.861352 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07704368-7dba-4bea-85b0-a54d41bd2fdb None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:11.861826 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07704368-7dba-4bea-85b0-a54d41bd2fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:11.862290 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07704368-7dba-4bea-85b0-a54d41bd2fdb None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:11.863624 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07704368-7dba-4bea-85b0-a54d41bd2fdb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:11.864155 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07704368-7dba-4bea-85b0-a54d41bd2fdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:11.865771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07704368-7dba-4bea-85b0-a54d41bd2fdb None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:11.871802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 316/628] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Oct 12 14:27:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:11.879729 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07704368-7dba-4bea-85b0-a54d41bd2fdb None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:11.944565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:11.968646 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:11.977327 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 313/629] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 14:27:12.031210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07704368-7dba-4bea-85b0-a54d41bd2fdb None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:12.071097 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04fd49-6fd8-402c-9f2d-24caad93c19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:12.072129 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04fd49-6fd8-402c-9f2d-24caad93c19a None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:12.072663 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04fd49-6fd8-402c-9f2d-24caad93c19a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:12.073132 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04fd49-6fd8-402c-9f2d-24caad93c19a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:12.074104 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04fd49-6fd8-402c-9f2d-24caad93c19a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:12.075685 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04fd49-6fd8-402c-9f2d-24caad93c19a None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:12.101637 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 317/630] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Oct 12 14:27:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:27:12.358676 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203799af-10c4-4ad9-ab03-0337fc7c4de8 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:12.470775 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203799af-10c4-4ad9-ab03-0337fc7c4de8 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:12.515016 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f82ee7-737b-46c7-800b-283fa318f590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:12.515679 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f82ee7-737b-46c7-800b-283fa318f590 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:12.515954 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f82ee7-737b-46c7-800b-283fa318f590 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:12.516223 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f82ee7-737b-46c7-800b-283fa318f590 None admin] PATH_INFO: `/v3/services/network` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:12.516798 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f82ee7-737b-46c7-800b-283fa318f590 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:12.518146 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f82ee7-737b-46c7-800b-283fa318f590 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-91f82ee7-737b-46c7-800b-283fa318f590 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:12.527459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:12.528434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:12.528434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:12.528434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:12.528434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:12.528434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:12.528434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:12.528434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:12.528434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:12.528434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:12.528434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:12.528434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:12.528434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:12.528434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:12.528434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:12.528434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:12.528434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:12.529244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:12.529244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:12.529244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:12.529244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:12.529244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:12.529244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:12.529244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:12.529244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:12.529244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:12.529244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:12.529244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:12.529244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:12.529244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 14:27:12.529244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:12.529244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 314/631] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:27:12 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:12.534731 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04fd49-6fd8-402c-9f2d-24caad93c19a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:12.640344 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:12.662906 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04fd49-6fd8-402c-9f2d-24caad93c19a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:12.707447 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5753a6-3dd9-4cc2-9b1d-126957b74270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:12.708050 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5753a6-3dd9-4cc2-9b1d-126957b74270 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:12.708327 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5753a6-3dd9-4cc2-9b1d-126957b74270 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:12.708598 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5753a6-3dd9-4cc2-9b1d-126957b74270 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:12.709221 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5753a6-3dd9-4cc2-9b1d-126957b74270 None admin] RBAC: Adding query filter params (name=network) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:12.709526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5753a6-3dd9-4cc2-9b1d-126957b74270 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:12.710941 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5753a6-3dd9-4cc2-9b1d-126957b74270 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:12.717490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 318/632] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Oct 12 14:27:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:12.723756 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f82ee7-737b-46c7-800b-283fa318f590 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:12.802422 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:12.864118 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f82ee7-737b-46c7-800b-283fa318f590 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:12.914647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee45c69-8fb0-498f-ba26-c90883924ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:12.915661 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee45c69-8fb0-498f-ba26-c90883924ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:12.916153 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee45c69-8fb0-498f-ba26-c90883924ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:12.916613 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee45c69-8fb0-498f-ba26-c90883924ca8 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:12.917455 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee45c69-8fb0-498f-ba26-c90883924ca8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:12.917944 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee45c69-8fb0-498f-ba26-c90883924ca8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:12.919533 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee45c69-8fb0-498f-ba26-c90883924ca8 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:12.927964 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 315/633] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Oct 12 14:27:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:12.933600 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5753a6-3dd9-4cc2-9b1d-126957b74270 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:13.073940 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5753a6-3dd9-4cc2-9b1d-126957b74270 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:13.119822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8171a9-b8ca-4427-b0cf-b0a541bc76f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:13.120428 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8171a9-b8ca-4427-b0cf-b0a541bc76f8 None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:13.120706 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8171a9-b8ca-4427-b0cf-b0a541bc76f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:13.120973 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8171a9-b8ca-4427-b0cf-b0a541bc76f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:13.121640 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8171a9-b8ca-4427-b0cf-b0a541bc76f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:13.123011 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8171a9-b8ca-4427-b0cf-b0a541bc76f8 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:13.146532 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 319/634] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Oct 12 14:27:12 2022] POST /identity/v3/endpoints => generated 337 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:27:13.477150 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5790b-e88d-40d9-97a7-14563aef5d40 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:13.477564 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5790b-e88d-40d9-97a7-14563aef5d40 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:13.477681 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5790b-e88d-40d9-97a7-14563aef5d40 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:13.479425 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 316/635] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:13.485456 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967941c3-244b-4f94-ab70-8bf196056510 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:13.485794 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967941c3-244b-4f94-ab70-8bf196056510 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:13.486000 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967941c3-244b-4f94-ab70-8bf196056510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:13.494255 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:13.509590 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-967941c3-244b-4f94-ab70-8bf196056510 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:13.511005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-967941c3-244b-4f94-ab70-8bf196056510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:13.607411 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79089a3-8ec7-4369-8115-f82a2d09068c None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:13.607763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79089a3-8ec7-4369-8115-f82a2d09068c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:13.607932 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79089a3-8ec7-4369-8115-f82a2d09068c None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:13.608159 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 320/636] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 14:27:13.609390 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 317/637] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:13.625236 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca035d-33d6-4658-873f-6b3773a92de5 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:13.625540 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca035d-33d6-4658-873f-6b3773a92de5 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:13.625775 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca035d-33d6-4658-873f-6b3773a92de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:13.645526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-b0ca035d-33d6-4658-873f-6b3773a92de5 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:13.646695 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-b0ca035d-33d6-4658-873f-6b3773a92de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:13.714995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 321/638] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:13.909379 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79089a3-8ec7-4369-8115-f82a2d09068c None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:13.986168 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca035d-33d6-4658-873f-6b3773a92de5 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:14.017755 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79089a3-8ec7-4369-8115-f82a2d09068c None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:14.042814 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:14.048690 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:14.059854 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3b1ea-eec5-4d76-84cc-4a44aaae9544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:14.060671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3b1ea-eec5-4d76-84cc-4a44aaae9544 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:14.061115 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3b1ea-eec5-4d76-84cc-4a44aaae9544 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:14.061524 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3b1ea-eec5-4d76-84cc-4a44aaae9544 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:14.062390 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3b1ea-eec5-4d76-84cc-4a44aaae9544 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:14.063982 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3b1ea-eec5-4d76-84cc-4a44aaae9544 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:14.083636 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 318/639] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Oct 12 14:27:13 2022] POST /identity/v3/services => generated 269 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:27:14.094014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca035d-33d6-4658-873f-6b3773a92de5 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:14.143535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a60db-027a-4392-84c8-cc7abbc1d1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:14.144167 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a60db-027a-4392-84c8-cc7abbc1d1af None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:14.144546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a60db-027a-4392-84c8-cc7abbc1d1af None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:14.144880 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a60db-027a-4392-84c8-cc7abbc1d1af None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:14.159804 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a60db-027a-4392-84c8-cc7abbc1d1af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:27:14.165879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-953a60db-027a-4392-84c8-cc7abbc1d1af None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 14:27:14.165879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:14.165879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:14.165879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:14.165879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:14.165879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:14.165879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:14.165879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:14.165879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:14.165879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:14.165879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:14.165879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:14.165879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:27:14.165879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:27:14.165879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:27:14.165879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:27:14.166856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:27:14.166856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:27:14.166856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:27:14.166856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:27:14.166856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:14.166856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:14.166856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:14.166856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:14.166856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:14.166856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:14.166856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:14.166856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:14.166856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:14.166856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:14.166856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:14.166856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:14.167687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:14.167687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:14.167687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:14.167687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:14.167687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:27:14.167687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:27:14.167687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:27:14.167687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:27:14.167687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:27:14.167687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:27:14.167687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 14:27:14.167687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:14.171549 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 322/640] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Oct 12 14:27:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:14.176622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3b1ea-eec5-4d76-84cc-4a44aaae9544 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:14.298089 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3b1ea-eec5-4d76-84cc-4a44aaae9544 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:14.337004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e64a7-fe0d-4a59-bba2-4bde696d8e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:14.337608 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e64a7-fe0d-4a59-bba2-4bde696d8e9b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:14.337880 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e64a7-fe0d-4a59-bba2-4bde696d8e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:14.338149 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e64a7-fe0d-4a59-bba2-4bde696d8e9b None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:14.338803 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e64a7-fe0d-4a59-bba2-4bde696d8e9b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:14.339097 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e64a7-fe0d-4a59-bba2-4bde696d8e9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:14.340484 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e64a7-fe0d-4a59-bba2-4bde696d8e9b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:14.350018 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 319/641] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Oct 12 14:27:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:14.355589 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a60db-027a-4392-84c8-cc7abbc1d1af None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:14.408872 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353805a6-6bd2-49bc-a821-9b2071224d6f None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:14.409156 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353805a6-6bd2-49bc-a821-9b2071224d6f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:14.409433 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353805a6-6bd2-49bc-a821-9b2071224d6f None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:14.411207 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 320/642] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:14.416189 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123ee58-96f3-47ef-ae0a-ab816588162c None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:14.416459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123ee58-96f3-47ef-ae0a-ab816588162c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:14.416744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123ee58-96f3-47ef-ae0a-ab816588162c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:14.435585 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3123ee58-96f3-47ef-ae0a-ab816588162c None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:14.436772 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3123ee58-96f3-47ef-ae0a-ab816588162c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:14.471847 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a60db-027a-4392-84c8-cc7abbc1d1af None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:14.479250 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:14.498487 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:14.523111 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 321/643] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:14 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 14:27:14.524144 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636d62d-3285-4141-94e5-453d8f68f2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:14.524777 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636d62d-3285-4141-94e5-453d8f68f2fc None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:14.525048 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636d62d-3285-4141-94e5-453d8f68f2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:14.525371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636d62d-3285-4141-94e5-453d8f68f2fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:14.540247 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636d62d-3285-4141-94e5-453d8f68f2fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:14.541985 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636d62d-3285-4141-94e5-453d8f68f2fc None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:14.558876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-1636d62d-3285-4141-94e5-453d8f68f2fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:14.558876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:14.558876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:14.558876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:14.558876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:14.558876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:14.558876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:14.558876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:14.558876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:14.558876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:14.558876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:14.558876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:14.558876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:14.558876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:14.558876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:14.558876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:14.559963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:14.560822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:14.560822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:14.560822 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:14.562858 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 323/644] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:14.568693 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123ee58-96f3-47ef-ae0a-ab816588162c None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:14.674201 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123ee58-96f3-47ef-ae0a-ab816588162c None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:14.711776 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582804e-8a44-4890-ba2e-d0facd0d6d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:14.712378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582804e-8a44-4890-ba2e-d0facd0d6d50 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:14.712647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582804e-8a44-4890-ba2e-d0facd0d6d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:14.712912 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582804e-8a44-4890-ba2e-d0facd0d6d50 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:14.713541 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582804e-8a44-4890-ba2e-d0facd0d6d50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:14.713884 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582804e-8a44-4890-ba2e-d0facd0d6d50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:14.715252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582804e-8a44-4890-ba2e-d0facd0d6d50 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:14.728107 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 322/645] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:14.733254 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636d62d-3285-4141-94e5-453d8f68f2fc None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:14.805615 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582804e-8a44-4890-ba2e-d0facd0d6d50 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:14.847127 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:14.851044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636d62d-3285-4141-94e5-453d8f68f2fc None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:14.887605 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90303b-d2d8-48c4-af87-b5a1e8e4fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:14.888230 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90303b-d2d8-48c4-af87-b5a1e8e4fdd6 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:14.888497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90303b-d2d8-48c4-af87-b5a1e8e4fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:14.888760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90303b-d2d8-48c4-af87-b5a1e8e4fdd6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:14.897134 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90303b-d2d8-48c4-af87-b5a1e8e4fdd6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:14.898496 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90303b-d2d8-48c4-af87-b5a1e8e4fdd6 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:14.910044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-8a90303b-d2d8-48c4-af87-b5a1e8e4fdd6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 14:27:14.910044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:14.910044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:14.910044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:14.910044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:14.910044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:14.910044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:14.910044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:14.910044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:14.910044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:14.910044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:14.910044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:14.910044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:27:14.910044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:27:14.910044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:27:14.910044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:14.911825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:14.912736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:14.912736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:14.912736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:27:14.912736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:27:14.912736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:27:14.912736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:27:14.912736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:27:14.912736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:27:14.912736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 14:27:14.912736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:14.917026 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 324/646] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Oct 12 14:27:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:14.917922 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90303b-d2d8-48c4-af87-b5a1e8e4fdd6 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:14.938214 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582804e-8a44-4890-ba2e-d0facd0d6d50 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:14.949176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:14.975991 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822befd4-957e-45f7-b672-3c5f9e8aba58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:14.977072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822befd4-957e-45f7-b672-3c5f9e8aba58 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:14.977349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822befd4-957e-45f7-b672-3c5f9e8aba58 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:14.977617 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822befd4-957e-45f7-b672-3c5f9e8aba58 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:14.978254 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822befd4-957e-45f7-b672-3c5f9e8aba58 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:14.979593 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822befd4-957e-45f7-b672-3c5f9e8aba58 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-822befd4-957e-45f7-b672-3c5f9e8aba58 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:14.988812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:14.990027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:14.990027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:14.990027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:14.990027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:14.990027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:14.990027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:14.990027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:14.990027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:14.990027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:14.990027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:14.990027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:14.990027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:14.990027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:14.990027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:14.990027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:14.990027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:14.990868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:14.990868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:14.990868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:14.990868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:14.990868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:14.990868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:14.990868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:14.990868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:14.990868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:14.990868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:14.990868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:14.990868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:14.990868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 14:27:14.990868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:14.990868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 323/647] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Oct 12 14:27:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:14.991807 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83d507-83b5-43aa-bb4b-3ea5b1331b5e None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:14.992084 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83d507-83b5-43aa-bb4b-3ea5b1331b5e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:14.995181 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83d507-83b5-43aa-bb4b-3ea5b1331b5e None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:14.995181 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 324/648] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:14.995729 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83d507-83b5-43aa-bb4b-3ea5b1331b5e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:15.043198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90303b-d2d8-48c4-af87-b5a1e8e4fdd6 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:15.080484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4204c6-2df0-4ebd-a7df-b096dcaac813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:15.081081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4204c6-2df0-4ebd-a7df-b096dcaac813 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:15.081393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4204c6-2df0-4ebd-a7df-b096dcaac813 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:15.081659 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4204c6-2df0-4ebd-a7df-b096dcaac813 None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:15.083021 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4204c6-2df0-4ebd-a7df-b096dcaac813 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:15.083359 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4204c6-2df0-4ebd-a7df-b096dcaac813 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:15.085071 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4204c6-2df0-4ebd-a7df-b096dcaac813 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:15.103128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 325/649] 10.222.0.16 () {56 vars in 1149 bytes} [Wed Oct 12 14:27:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:15.104400 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e5930-aa20-465e-8891-6e1318071f0f None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:15.104624 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e5930-aa20-465e-8891-6e1318071f0f None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:15.104895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e5930-aa20-465e-8891-6e1318071f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:15.111219 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:15.118424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83d507-83b5-43aa-bb4b-3ea5b1331b5e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:15.125082 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-b94e5930-aa20-465e-8891-6e1318071f0f None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:15.126296 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-b94e5930-aa20-465e-8891-6e1318071f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:15.158923 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09376839-0e0b-4062-9c33-5663f103922f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:15.159614 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09376839-0e0b-4062-9c33-5663f103922f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:15.159893 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09376839-0e0b-4062-9c33-5663f103922f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:15.160252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09376839-0e0b-4062-9c33-5663f103922f None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:15.161068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09376839-0e0b-4062-9c33-5663f103922f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:15.161365 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09376839-0e0b-4062-9c33-5663f103922f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:15.162860 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09376839-0e0b-4062-9c33-5663f103922f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:15.171179 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 325/650] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Oct 12 14:27:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:15.171179 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09376839-0e0b-4062-9c33-5663f103922f None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:15.200514 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 326/651] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:15 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:15.201361 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e5930-aa20-465e-8891-6e1318071f0f None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:15.226281 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:15.289411 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09376839-0e0b-4062-9c33-5663f103922f None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:15.326618 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f0bfa-6ea0-4f1e-948b-37de7ede391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:15.327607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f0bfa-6ea0-4f1e-948b-37de7ede391d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:15.328091 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f0bfa-6ea0-4f1e-948b-37de7ede391d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:15.328332 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e5930-aa20-465e-8891-6e1318071f0f None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:15.328802 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f0bfa-6ea0-4f1e-948b-37de7ede391d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:15.343818 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f0bfa-6ea0-4f1e-948b-37de7ede391d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:15.345632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f0bfa-6ea0-4f1e-948b-37de7ede391d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:15.361842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-ea7f0bfa-6ea0-4f1e-948b-37de7ede391d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:15.361842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:15.361842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:15.361842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:15.361842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:15.361842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:15.361842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:15.361842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:15.361842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:15.361842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:15.361842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:15.361842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:15.361842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:15.361842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:15.361842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:15.361842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:15.362809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:15.363773 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:15.363773 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:15.363773 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:15.365654 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 326/652] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:15.370381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10cf09-b77b-4226-960a-88e4eb97841d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:15.370381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10cf09-b77b-4226-960a-88e4eb97841d None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:15.370381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10cf09-b77b-4226-960a-88e4eb97841d None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:15.370381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10cf09-b77b-4226-960a-88e4eb97841d None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:15.371057 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f0bfa-6ea0-4f1e-948b-37de7ede391d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:15.371663 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10cf09-b77b-4226-960a-88e4eb97841d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:15.371959 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10cf09-b77b-4226-960a-88e4eb97841d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:15.373312 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10cf09-b77b-4226-960a-88e4eb97841d None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:15.385049 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 327/653] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Oct 12 14:27:15 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:15.391973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:15.489843 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f0bfa-6ea0-4f1e-948b-37de7ede391d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:15.507117 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10cf09-b77b-4226-960a-88e4eb97841d None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:15.530068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04667c-6a20-4b26-b500-cbb4bd41536d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:15.531218 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04667c-6a20-4b26-b500-cbb4bd41536d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:15.531712 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04667c-6a20-4b26-b500-cbb4bd41536d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:15.532306 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04667c-6a20-4b26-b500-cbb4bd41536d None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:15.532974 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04667c-6a20-4b26-b500-cbb4bd41536d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:15.533278 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04667c-6a20-4b26-b500-cbb4bd41536d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:15.534673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04667c-6a20-4b26-b500-cbb4bd41536d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:15.549049 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 327/654] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:15.554828 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04667c-6a20-4b26-b500-cbb4bd41536d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:15.608899 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:15.641818 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:15.648561 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10cf09-b77b-4226-960a-88e4eb97841d None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:15.673579 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04667c-6a20-4b26-b500-cbb4bd41536d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:15.697382 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333abd1-9209-4887-92c0-6c482b5e0f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:15.698872 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333abd1-9209-4887-92c0-6c482b5e0f32 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:15.699672 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333abd1-9209-4887-92c0-6c482b5e0f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:15.700157 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333abd1-9209-4887-92c0-6c482b5e0f32 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:15.701090 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333abd1-9209-4887-92c0-6c482b5e0f32 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:15.703285 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333abd1-9209-4887-92c0-6c482b5e0f32 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:15.709664 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46046b8d-0397-4db6-a175-7f5191f0d78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:15.710292 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46046b8d-0397-4db6-a175-7f5191f0d78c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:15.710568 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46046b8d-0397-4db6-a175-7f5191f0d78c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:15.710835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46046b8d-0397-4db6-a175-7f5191f0d78c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-6333abd1-9209-4887-92c0-6c482b5e0f32 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:15.715019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:15.715997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:15.715997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:15.715997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:15.715997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:15.715997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:15.715997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:15.715997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:15.715997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:15.715997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:15.715997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:15.715997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:15.715997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:15.715997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:15.715997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:15.715997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:15.715997 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:15.716861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:15.716861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:15.716861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:15.716861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:15.716861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:15.716861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:15.716861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:15.716861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:15.716861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:15.716861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:15.716861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:15.716861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:15.716861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 14:27:15.716861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:15.719074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46046b8d-0397-4db6-a175-7f5191f0d78c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:15.723620 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46046b8d-0397-4db6-a175-7f5191f0d78c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:15.723743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 328/655] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Oct 12 14:27:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:15.726607 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333abd1-9209-4887-92c0-6c482b5e0f32 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:15.736392 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-46046b8d-0397-4db6-a175-7f5191f0d78c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:15.736392 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:15.736392 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:15.736392 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:15.736392 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:15.736392 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:15.736392 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:15.736392 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:15.736392 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:15.736392 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:15.736392 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:15.736392 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:15.736392 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:27:15.736392 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:27:15.736392 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:15.737276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:15.738181 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:27:15.738181 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:27:15.738181 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:27:15.738181 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:27:15.738181 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:27:15.738181 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:27:15.738181 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:15.738181 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:15.738181 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 328/656] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:27:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:15.744370 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46046b8d-0397-4db6-a175-7f5191f0d78c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:15.853266 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46046b8d-0397-4db6-a175-7f5191f0d78c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:15.858790 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333abd1-9209-4887-92c0-6c482b5e0f32 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:15.893919 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:15.897912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419c35f-70a3-4602-b66c-646becef226c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:15.898731 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419c35f-70a3-4602-b66c-646becef226c None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:15.899185 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419c35f-70a3-4602-b66c-646becef226c None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:15.899645 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419c35f-70a3-4602-b66c-646becef226c None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:15.901235 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419c35f-70a3-4602-b66c-646becef226c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:15.902914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224d6e6-7438-4020-87c7-a9602d4b4557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:15.902914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224d6e6-7438-4020-87c7-a9602d4b4557 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:15.902914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224d6e6-7438-4020-87c7-a9602d4b4557 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:15.903317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224d6e6-7438-4020-87c7-a9602d4b4557 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:15.903713 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419c35f-70a3-4602-b66c-646becef226c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:15.904188 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224d6e6-7438-4020-87c7-a9602d4b4557 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:15.904641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224d6e6-7438-4020-87c7-a9602d4b4557 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:15.905659 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419c35f-70a3-4602-b66c-646becef226c None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:15.906258 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224d6e6-7438-4020-87c7-a9602d4b4557 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:15.911436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 329/657] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Oct 12 14:27:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:15.919895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419c35f-70a3-4602-b66c-646becef226c None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:15.930661 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 329/658] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:27:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:15.931599 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224d6e6-7438-4020-87c7-a9602d4b4557 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:15.935117 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:16.032740 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419c35f-70a3-4602-b66c-646becef226c None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:16.036401 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224d6e6-7438-4020-87c7-a9602d4b4557 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:16.069944 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba046d-43a7-4d10-8512-009fa73c86d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:16.070946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba046d-43a7-4d10-8512-009fa73c86d0 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:16.071488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba046d-43a7-4d10-8512-009fa73c86d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:16.071921 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba046d-43a7-4d10-8512-009fa73c86d0 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:16.072706 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba046d-43a7-4d10-8512-009fa73c86d0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:16.073238 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba046d-43a7-4d10-8512-009fa73c86d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:16.073703 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00b533-b833-44ed-aa93-809fecface9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:16.074470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00b533-b833-44ed-aa93-809fecface9f None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:16.074846 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00b533-b833-44ed-aa93-809fecface9f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:16.075260 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00b533-b833-44ed-aa93-809fecface9f None admin] PATH_INFO: `/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/15d7cb817428475e80cf697ac94c62b2/roles/f074c279fe6544eba0c9d54f5150fbd6` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:16.075445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba046d-43a7-4d10-8512-009fa73c86d0 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:16.077742 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00b533-b833-44ed-aa93-809fecface9f None admin] RBAC: Authorizing `identity:create_grant(project_id=720b11237d2e48128c171be45fcd0ad0, user_id=15d7cb817428475e80cf697ac94c62b2, role_id=f074c279fe6544eba0c9d54f5150fbd6)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:16.079330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00b533-b833-44ed-aa93-809fecface9f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:16.085089 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 330/659] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Oct 12 14:27:15 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:16.095951 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 330/660] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:27:15 2022] PUT /identity/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/15d7cb817428475e80cf697ac94c62b2/roles/f074c279fe6544eba0c9d54f5150fbd6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:27:16.486985 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e2d85-7329-482e-a87b-2b8bc35c0904 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:16.487526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e2d85-7329-482e-a87b-2b8bc35c0904 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:16.488129 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e2d85-7329-482e-a87b-2b8bc35c0904 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:16.490982 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 331/661] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:16.499303 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da5fcf-41c6-43e3-bb87-19a704c5ea9a None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:16.499763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da5fcf-41c6-43e3-bb87-19a704c5ea9a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:16.500100 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da5fcf-41c6-43e3-bb87-19a704c5ea9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:16.521185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-48da5fcf-41c6-43e3-bb87-19a704c5ea9a None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:16.522992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-48da5fcf-41c6-43e3-bb87-19a704c5ea9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:16.592377 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:16.618353 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 331/662] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:16 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:16.914458 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e2d85-7329-482e-a87b-2b8bc35c0904 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:16.998494 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:17.039837 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e2d85-7329-482e-a87b-2b8bc35c0904 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:17.075995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa88ed-d72d-4bf3-b82b-81c8ee957087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:17.076628 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa88ed-d72d-4bf3-b82b-81c8ee957087 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:17.076900 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa88ed-d72d-4bf3-b82b-81c8ee957087 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:17.077166 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa88ed-d72d-4bf3-b82b-81c8ee957087 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:17.077746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa88ed-d72d-4bf3-b82b-81c8ee957087 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:17.079124 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa88ed-d72d-4bf3-b82b-81c8ee957087 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:17.089874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-6caa88ed-d72d-4bf3-b82b-81c8ee957087 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 14:27:17.089874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:17.089874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:17.089874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:17.089874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:17.089874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:17.089874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:17.089874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:17.089874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 14:27:17.089874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:17.089874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:17.089874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:17.089874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:17.089874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:17.089874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:17.089874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:17.091812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:17.092887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:17.092887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:17.092887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:17.092887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:17.092887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:17.092887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:17.092887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:17.092887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:17.092887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:17.092887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:17.092887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:17.092887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:17.092887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 14:27:17.092887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:17.092887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 332/663] 10.222.0.16 () {56 vars in 1132 bytes} [Wed Oct 12 14:27:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:17.101268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da5fcf-41c6-43e3-bb87-19a704c5ea9a None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:17.216939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da5fcf-41c6-43e3-bb87-19a704c5ea9a None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:17.253525 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241b99c-4705-49c7-beb9-58a8946938bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:17.254155 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241b99c-4705-49c7-beb9-58a8946938bb None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:17.254436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241b99c-4705-49c7-beb9-58a8946938bb None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:17.254727 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241b99c-4705-49c7-beb9-58a8946938bb None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:17.255534 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241b99c-4705-49c7-beb9-58a8946938bb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:17.255832 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241b99c-4705-49c7-beb9-58a8946938bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:17.257261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241b99c-4705-49c7-beb9-58a8946938bb None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:17.262343 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 332/664] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Oct 12 14:27:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:17.271707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa88ed-d72d-4bf3-b82b-81c8ee957087 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:17.376664 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa88ed-d72d-4bf3-b82b-81c8ee957087 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:17.414314 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8331b-73fa-43e9-a454-1ac4281dfd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:17.414916 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8331b-73fa-43e9-a454-1ac4281dfd48 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:17.415216 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8331b-73fa-43e9-a454-1ac4281dfd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:17.415486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8331b-73fa-43e9-a454-1ac4281dfd48 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:17.416109 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8331b-73fa-43e9-a454-1ac4281dfd48 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:17.416406 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8331b-73fa-43e9-a454-1ac4281dfd48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:17.417765 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8331b-73fa-43e9-a454-1ac4281dfd48 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:17.419887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:17.427474 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 333/665] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Oct 12 14:27:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:17.430116 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241b99c-4705-49c7-beb9-58a8946938bb None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:17.483608 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:17.541577 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241b99c-4705-49c7-beb9-58a8946938bb None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:17.591952 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2c34a-78ff-4b68-b24c-62aa0386a35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:17.592573 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2c34a-78ff-4b68-b24c-62aa0386a35a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:17.592854 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2c34a-78ff-4b68-b24c-62aa0386a35a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:17.593178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2c34a-78ff-4b68-b24c-62aa0386a35a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:17.593881 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2c34a-78ff-4b68-b24c-62aa0386a35a None admin] RBAC: Adding query filter params (interface=public, service_id=66bb38e422bf49a2a9c5fc7bdd9b3741, region_id=RegionOne) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:17.594184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2c34a-78ff-4b68-b24c-62aa0386a35a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:17.595582 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2c34a-78ff-4b68-b24c-62aa0386a35a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:17.602197 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 333/666] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Oct 12 14:27:17 2022] GET /identity/v3/endpoints?service_id=66bb38e422bf49a2a9c5fc7bdd9b3741&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:17.608437 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8331b-73fa-43e9-a454-1ac4281dfd48 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:17.630973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cd219-7bb9-448e-8a70-ba52152292c6 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:17.631318 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cd219-7bb9-448e-8a70-ba52152292c6 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:17.631597 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cd219-7bb9-448e-8a70-ba52152292c6 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:17.633039 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 334/667] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:27:17.638066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43074f69-506e-4553-8a04-d3393ffd61b2 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:17.638398 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43074f69-506e-4553-8a04-d3393ffd61b2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:17.638671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43074f69-506e-4553-8a04-d3393ffd61b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:17.663423 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-43074f69-506e-4553-8a04-d3393ffd61b2 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:17.664621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-43074f69-506e-4553-8a04-d3393ffd61b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:17.724162 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8331b-73fa-43e9-a454-1ac4281dfd48 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:17.759270 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 335/668] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:17 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 14:27:17.763824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1faff-779d-4100-89dc-efcc81d08468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:17.764600 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1faff-779d-4100-89dc-efcc81d08468 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:17.765013 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1faff-779d-4100-89dc-efcc81d08468 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:17.765422 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1faff-779d-4100-89dc-efcc81d08468 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:17.766167 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1faff-779d-4100-89dc-efcc81d08468 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:17.767701 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1faff-779d-4100-89dc-efcc81d08468 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:17.774099 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 334/669] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Oct 12 14:27:17 2022] GET /identity/v3/services => generated 1029 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 14:27:18.036400 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43074f69-506e-4553-8a04-d3393ffd61b2 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:18.079306 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:18.139910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43074f69-506e-4553-8a04-d3393ffd61b2 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:18.175358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28f101-903f-46f1-8cee-cd5c43fc3d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:18.177020 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28f101-903f-46f1-8cee-cd5c43fc3d2f None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:18.177296 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28f101-903f-46f1-8cee-cd5c43fc3d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:18.177561 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28f101-903f-46f1-8cee-cd5c43fc3d2f None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:18.178226 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28f101-903f-46f1-8cee-cd5c43fc3d2f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:18.179634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28f101-903f-46f1-8cee-cd5c43fc3d2f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:18.195655 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 336/670] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Oct 12 14:27:18 2022] POST /identity/v3/services => generated 241 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:27:18.460431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06beb8-ec44-4a4f-89b7-65c52dd368d7 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:18.460949 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06beb8-ec44-4a4f-89b7-65c52dd368d7 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:18.460949 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06beb8-ec44-4a4f-89b7-65c52dd368d7 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:18.462479 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 335/671] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:18.468037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73172d-ab09-4092-83ab-1aaa7f0eaec8 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:18.468579 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73172d-ab09-4092-83ab-1aaa7f0eaec8 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:18.469051 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73172d-ab09-4092-83ab-1aaa7f0eaec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:18.482499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba6c09-765b-4d8b-bb07-09038ce1a329 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:18.483094 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba6c09-765b-4d8b-bb07-09038ce1a329 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:18.483621 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba6c09-765b-4d8b-bb07-09038ce1a329 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:18.485457 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 336/672] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:27:18.490901 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56ed64-36fb-4d7b-af6b-759bfe26fb47 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:18.491195 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56ed64-36fb-4d7b-af6b-759bfe26fb47 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:18.491467 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56ed64-36fb-4d7b-af6b-759bfe26fb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:18.492810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ae73172d-ab09-4092-83ab-1aaa7f0eaec8 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:18.494236 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ae73172d-ab09-4092-83ab-1aaa7f0eaec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:18.510116 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-cd56ed64-36fb-4d7b-af6b-759bfe26fb47 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:18.511344 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-cd56ed64-36fb-4d7b-af6b-759bfe26fb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:18.582214 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:18.591357 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 337/674] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:18 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 14:27:18.591676 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 337/674] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:18 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:18.858647 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56ed64-36fb-4d7b-af6b-759bfe26fb47 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:18.858819 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73172d-ab09-4092-83ab-1aaa7f0eaec8 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:18.897606 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:18.968280 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56ed64-36fb-4d7b-af6b-759bfe26fb47 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:18.994727 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73172d-ab09-4092-83ab-1aaa7f0eaec8 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:19.008798 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d83811-2f8c-4303-be0b-11d323b3a323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:19.009440 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d83811-2f8c-4303-be0b-11d323b3a323 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:19.009709 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d83811-2f8c-4303-be0b-11d323b3a323 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:19.009974 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d83811-2f8c-4303-be0b-11d323b3a323 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:19.021882 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d83811-2f8c-4303-be0b-11d323b3a323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:27:19.027526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-80d83811-2f8c-4303-be0b-11d323b3a323 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 14:27:19.027526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:19.027526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:19.027526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:19.027526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:19.027526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:19.027526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:19.027526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:19.027526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:19.027526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:19.027526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:19.027526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:19.027526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:27:19.027526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:27:19.027526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:27:19.027526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:27:19.028421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:27:19.028421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:27:19.028421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:27:19.028421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:27:19.028421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:19.028421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:19.028421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:19.028421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:19.028421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:19.028421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:19.028421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:19.028421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:19.028421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:19.028421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:19.028421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:19.028421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:19.029423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:19.029423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:19.029423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:19.029423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:19.029423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:27:19.029423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:27:19.029423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:27:19.029423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:27:19.029423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:27:19.029423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:27:19.029423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 14:27:19.029423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:19.029423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 338/675] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Oct 12 14:27:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:19.036294 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d83811-2f8c-4303-be0b-11d323b3a323 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:19.044682 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd50f7-e1bd-44e7-bf3f-9d174133cdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:19.045580 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd50f7-e1bd-44e7-bf3f-9d174133cdce None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:19.046033 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd50f7-e1bd-44e7-bf3f-9d174133cdce None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:19.046479 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd50f7-e1bd-44e7-bf3f-9d174133cdce None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:19.047350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd50f7-e1bd-44e7-bf3f-9d174133cdce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:19.048854 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd50f7-e1bd-44e7-bf3f-9d174133cdce None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:19.066017 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 338/676] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Oct 12 14:27:18 2022] POST /identity/v3/services => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 14:27:19.113108 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:19.175557 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d83811-2f8c-4303-be0b-11d323b3a323 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:19.222032 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66161e9-c8de-49ac-b46a-6b5aff882c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:19.222645 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66161e9-c8de-49ac-b46a-6b5aff882c60 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:19.222917 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66161e9-c8de-49ac-b46a-6b5aff882c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:19.223772 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66161e9-c8de-49ac-b46a-6b5aff882c60 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:19.224891 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66161e9-c8de-49ac-b46a-6b5aff882c60 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:19.225203 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66161e9-c8de-49ac-b46a-6b5aff882c60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:19.226686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66161e9-c8de-49ac-b46a-6b5aff882c60 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:19.236290 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 339/677] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Oct 12 14:27:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:19.242427 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd50f7-e1bd-44e7-bf3f-9d174133cdce None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:19.332027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:19.352418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd50f7-e1bd-44e7-bf3f-9d174133cdce None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:19.389716 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10544275-f36b-4148-b440-913fe500e556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:19.390320 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10544275-f36b-4148-b440-913fe500e556 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:19.390596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10544275-f36b-4148-b440-913fe500e556 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:19.390861 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10544275-f36b-4148-b440-913fe500e556 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:19.403300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10544275-f36b-4148-b440-913fe500e556 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:19.404732 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10544275-f36b-4148-b440-913fe500e556 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:19.420624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-10544275-f36b-4148-b440-913fe500e556 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:19.420624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:19.420624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:19.420624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:19.420624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:19.420624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:19.420624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:19.420624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:19.420624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:19.420624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:19.420624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:19.420624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:19.420624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:19.420624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:19.420624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:19.420624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:19.421572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:19.422593 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:19.422593 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:19.422593 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:19.422593 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 339/678] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:19.428998 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66161e9-c8de-49ac-b46a-6b5aff882c60 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:19.537142 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66161e9-c8de-49ac-b46a-6b5aff882c60 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:19.550644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:19.585058 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba6796-fe8c-405a-bd8e-f20c894ec37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:19.585683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba6796-fe8c-405a-bd8e-f20c894ec37e None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:19.585958 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba6796-fe8c-405a-bd8e-f20c894ec37e None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:19.586231 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba6796-fe8c-405a-bd8e-f20c894ec37e None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:19.586875 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba6796-fe8c-405a-bd8e-f20c894ec37e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:19.587193 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba6796-fe8c-405a-bd8e-f20c894ec37e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:19.588569 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba6796-fe8c-405a-bd8e-f20c894ec37e None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:19.601735 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 340/679] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:19.606998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10544275-f36b-4148-b440-913fe500e556 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:19.710903 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10544275-f36b-4148-b440-913fe500e556 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:19.745793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d78fcd-36a5-4e27-b97e-d8eb66a45f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:19.746752 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d78fcd-36a5-4e27-b97e-d8eb66a45f59 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:19.747261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d78fcd-36a5-4e27-b97e-d8eb66a45f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:19.747789 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d78fcd-36a5-4e27-b97e-d8eb66a45f59 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:19.755663 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d78fcd-36a5-4e27-b97e-d8eb66a45f59 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:19.757328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d78fcd-36a5-4e27-b97e-d8eb66a45f59 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:19.759261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:19.769171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-75d78fcd-36a5-4e27-b97e-d8eb66a45f59 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 14:27:19.769171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:19.769171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:19.769171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:19.769171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:19.769171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:19.769171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:19.769171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:19.769171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:19.769171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:19.769171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:19.769171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:19.769171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:27:19.769171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:27:19.769171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:27:19.769171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:19.771034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:19.772267 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:19.772267 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:19.772267 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:27:19.772267 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:27:19.772267 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:27:19.772267 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:27:19.772267 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:27:19.772267 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:27:19.772267 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 14:27:19.772267 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:19.773488 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 340/680] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Oct 12 14:27:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:19.778592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba6796-fe8c-405a-bd8e-f20c894ec37e None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:19.889148 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba6796-fe8c-405a-bd8e-f20c894ec37e None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:19.926593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6723e3b-88a8-44c6-9f62-ea278140a757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:19.927629 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6723e3b-88a8-44c6-9f62-ea278140a757 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:19.928105 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6723e3b-88a8-44c6-9f62-ea278140a757 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:19.928566 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6723e3b-88a8-44c6-9f62-ea278140a757 None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:19.929496 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6723e3b-88a8-44c6-9f62-ea278140a757 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:19.929995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6723e3b-88a8-44c6-9f62-ea278140a757 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:19.931655 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6723e3b-88a8-44c6-9f62-ea278140a757 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:19.951089 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 341/681] 10.222.0.16 () {56 vars in 1149 bytes} [Wed Oct 12 14:27:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:19.956621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d78fcd-36a5-4e27-b97e-d8eb66a45f59 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:19.988477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3ed29-38d7-4c64-bfad-675522faa265 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:19.989033 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3ed29-38d7-4c64-bfad-675522faa265 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:19.989684 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3ed29-38d7-4c64-bfad-675522faa265 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:19.991603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 342/682] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:19.996309 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f51a2-9073-4f9b-bd38-c05911071c4d None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:19.996585 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f51a2-9073-4f9b-bd38-c05911071c4d None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:19.996850 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f51a2-9073-4f9b-bd38-c05911071c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:20.015179 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-786f51a2-9073-4f9b-bd38-c05911071c4d None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:20.016671 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-786f51a2-9073-4f9b-bd38-c05911071c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:20.042715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:20.070271 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d78fcd-36a5-4e27-b97e-d8eb66a45f59 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:20.091121 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 343/683] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 14:27:20.112507 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b72ef2-083d-4482-8706-797b8dd21421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:20.113389 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b72ef2-083d-4482-8706-797b8dd21421 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:20.113811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b72ef2-083d-4482-8706-797b8dd21421 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:20.114473 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b72ef2-083d-4482-8706-797b8dd21421 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:20.180509 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:20.184285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b72ef2-083d-4482-8706-797b8dd21421 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:20.186067 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b72ef2-083d-4482-8706-797b8dd21421 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:20.292160 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 341/684] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:27:19 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 14:27:20.298066 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f51a2-9073-4f9b-bd38-c05911071c4d None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:20.372463 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b72ef2-083d-4482-8706-797b8dd21421 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:20.412333 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f51a2-9073-4f9b-bd38-c05911071c4d None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:20.460527 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684a7fc-4948-474d-812d-652f770abeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:20.461492 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684a7fc-4948-474d-812d-652f770abeac None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:20.461763 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684a7fc-4948-474d-812d-652f770abeac None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:20.462053 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684a7fc-4948-474d-812d-652f770abeac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:20.476906 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684a7fc-4948-474d-812d-652f770abeac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:20.479350 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684a7fc-4948-474d-812d-652f770abeac None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:20.500561 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:20.502849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-d684a7fc-4948-474d-812d-652f770abeac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:20.502849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:20.502849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:20.502849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:20.502849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:20.502849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:20.502849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:20.502849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:20.502849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:20.502849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:20.502849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:20.502849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:20.502849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:20.502849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:20.502849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:20.502849 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:20.504024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:20.505620 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:20.505620 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:20.505620 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:20.505620 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 344/685] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:20.505905 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b72ef2-083d-4482-8706-797b8dd21421 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:20.509971 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684a7fc-4948-474d-812d-652f770abeac None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:20.531710 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:20.554930 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd8d8d-f2ff-44ac-bce9-34cc7d806705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:20.555513 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd8d8d-f2ff-44ac-bce9-34cc7d806705 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:20.555782 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd8d8d-f2ff-44ac-bce9-34cc7d806705 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:20.556047 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd8d8d-f2ff-44ac-bce9-34cc7d806705 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:20.556625 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd8d8d-f2ff-44ac-bce9-34cc7d806705 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:20.557981 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd8d8d-f2ff-44ac-bce9-34cc7d806705 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:20.569657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-d7fd8d8d-f2ff-44ac-bce9-34cc7d806705 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 14:27:20.569657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:20.569657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:20.569657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:20.569657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:20.569657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:20.569657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:20.569657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:20.569657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 14:27:20.569657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:20.569657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:20.569657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:20.569657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:20.569657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:20.569657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:20.569657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:20.570622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:20.571532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:20.571532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:20.571532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:20.571532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:20.571532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:20.571532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:20.571532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:20.571532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:20.571532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:20.571532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:20.571532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:20.571532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:20.571532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 14:27:20.571532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:20.571532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 342/686] 10.222.0.16 () {56 vars in 1132 bytes} [Wed Oct 12 14:27:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:20.578610 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2c5ff-aeb1-4225-8703-6b2613149b46 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:20.578887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2c5ff-aeb1-4225-8703-6b2613149b46 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:20.579171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2c5ff-aeb1-4225-8703-6b2613149b46 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:20.580862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 343/687] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:20.581561 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2c5ff-aeb1-4225-8703-6b2613149b46 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:20.633548 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684a7fc-4948-474d-812d-652f770abeac None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:20.668324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75884a44-18bb-4468-9e7e-b14932b3e3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:20.668918 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75884a44-18bb-4468-9e7e-b14932b3e3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:20.669186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75884a44-18bb-4468-9e7e-b14932b3e3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:20.669653 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75884a44-18bb-4468-9e7e-b14932b3e3b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:20.670063 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75884a44-18bb-4468-9e7e-b14932b3e3b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:20.670357 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75884a44-18bb-4468-9e7e-b14932b3e3b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:20.671789 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75884a44-18bb-4468-9e7e-b14932b3e3b7 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:20.684098 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 345/688] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:20.685078 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1e786-b6be-450b-8116-0fa4eba2b413 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:20.685354 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1e786-b6be-450b-8116-0fa4eba2b413 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:20.685616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1e786-b6be-450b-8116-0fa4eba2b413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:20.700694 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2c5ff-aeb1-4225-8703-6b2613149b46 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:20.703310 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-c6d1e786-b6be-450b-8116-0fa4eba2b413 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:20.704463 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-c6d1e786-b6be-450b-8116-0fa4eba2b413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:20.738555 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c88be5-d47d-4682-9bbf-ddaaccce544c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:20.739182 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c88be5-d47d-4682-9bbf-ddaaccce544c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:20.739472 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c88be5-d47d-4682-9bbf-ddaaccce544c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:20.739831 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c88be5-d47d-4682-9bbf-ddaaccce544c None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:20.740461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c88be5-d47d-4682-9bbf-ddaaccce544c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:20.740760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c88be5-d47d-4682-9bbf-ddaaccce544c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:20.742117 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c88be5-d47d-4682-9bbf-ddaaccce544c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:20.751231 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 344/689] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Oct 12 14:27:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:20.751231 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c88be5-d47d-4682-9bbf-ddaaccce544c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:20.778561 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 346/690] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:20.779434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1e786-b6be-450b-8116-0fa4eba2b413 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:20.781558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:20.789456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:20.885887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c88be5-d47d-4682-9bbf-ddaaccce544c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:20.912685 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1e786-b6be-450b-8116-0fa4eba2b413 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:20.929268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d0865-750d-45f0-91c3-f6027325aa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:20.930445 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d0865-750d-45f0-91c3-f6027325aa4e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:20.930926 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d0865-750d-45f0-91c3-f6027325aa4e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:20.931866 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d0865-750d-45f0-91c3-f6027325aa4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:20.940708 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d0865-750d-45f0-91c3-f6027325aa4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:20.943603 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d0865-750d-45f0-91c3-f6027325aa4e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:20.954919 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff3ca0-0181-4281-83ce-1913aa112762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:20.955622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff3ca0-0181-4281-83ce-1913aa112762 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:20.955960 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff3ca0-0181-4281-83ce-1913aa112762 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:20.956292 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff3ca0-0181-4281-83ce-1913aa112762 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:20.956979 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff3ca0-0181-4281-83ce-1913aa112762 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:20.957341 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff3ca0-0181-4281-83ce-1913aa112762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:20.959358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-0a9d0865-750d-45f0-91c3-f6027325aa4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:20.959358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:20.959358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:20.959358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:20.959358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:20.959358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:20.959358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:20.959358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:20.959358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:20.959358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:20.959358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:20.959358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:20.959358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:27:20.959358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:27:20.959358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:20.960222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:20.961091 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:27:20.961091 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:27:20.961091 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:27:20.961091 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:27:20.961091 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:27:20.961091 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:27:20.961091 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:20.961091 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:20.961510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff3ca0-0181-4281-83ce-1913aa112762 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:20.963991 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 345/691] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:27:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:20.969459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d0865-750d-45f0-91c3-f6027325aa4e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:20.970486 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 347/692] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Oct 12 14:27:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:20.978337 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff3ca0-0181-4281-83ce-1913aa112762 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:21.067753 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:21.084763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:21.101713 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d0865-750d-45f0-91c3-f6027325aa4e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:21.105627 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff3ca0-0181-4281-83ce-1913aa112762 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:21.140199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073ee24-dff4-4c56-b4dc-0707c5659735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:21.140725 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073ee24-dff4-4c56-b4dc-0707c5659735 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:21.141001 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073ee24-dff4-4c56-b4dc-0707c5659735 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:21.141240 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b38be3-dea8-4f63-a825-8ef1f49f1a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:21.141355 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073ee24-dff4-4c56-b4dc-0707c5659735 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:21.141838 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b38be3-dea8-4f63-a825-8ef1f49f1a0c None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:21.141952 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073ee24-dff4-4c56-b4dc-0707c5659735 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:21.142106 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b38be3-dea8-4f63-a825-8ef1f49f1a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:21.142258 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073ee24-dff4-4c56-b4dc-0707c5659735 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:21.142369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b38be3-dea8-4f63-a825-8ef1f49f1a0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:21.143016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b38be3-dea8-4f63-a825-8ef1f49f1a0c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:21.143702 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073ee24-dff4-4c56-b4dc-0707c5659735 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:21.144390 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b38be3-dea8-4f63-a825-8ef1f49f1a0c None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:21.157467 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 346/693] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:27:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:21.158558 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073ee24-dff4-4c56-b4dc-0707c5659735 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:21.169586 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 348/694] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Oct 12 14:27:20 2022] POST /identity/v3/endpoints => generated 347 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:27:21.170280 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b38be3-dea8-4f63-a825-8ef1f49f1a0c None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:21.294586 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b38be3-dea8-4f63-a825-8ef1f49f1a0c None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:21.311348 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073ee24-dff4-4c56-b4dc-0707c5659735 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:21.317065 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:21.339667 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac939cd-d1d1-4f3f-a428-0c4b51cd5d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:21.340331 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac939cd-d1d1-4f3f-a428-0c4b51cd5d49 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:21.340616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac939cd-d1d1-4f3f-a428-0c4b51cd5d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:21.340946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac939cd-d1d1-4f3f-a428-0c4b51cd5d49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:21.341677 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac939cd-d1d1-4f3f-a428-0c4b51cd5d49 None admin] RBAC: Adding query filter params (role.id=f074c279fe6544eba0c9d54f5150fbd6, scope.project.id=720b11237d2e48128c171be45fcd0ad0, user.id=15d7cb817428475e80cf697ac94c62b2) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:21.342205 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac939cd-d1d1-4f3f-a428-0c4b51cd5d49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:21.343462 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac939cd-d1d1-4f3f-a428-0c4b51cd5d49 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:21.355569 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 349/695] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:27:21 2022] GET /identity/v3/role_assignments?user.id=15d7cb817428475e80cf697ac94c62b2&scope.project.id=720b11237d2e48128c171be45fcd0ad0&role.id=f074c279fe6544eba0c9d54f5150fbd6 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:21.362069 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cff584-63fe-4ed0-9e3c-4fdaa833df53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:21.362727 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cff584-63fe-4ed0-9e3c-4fdaa833df53 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:21.363000 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cff584-63fe-4ed0-9e3c-4fdaa833df53 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:21.363300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cff584-63fe-4ed0-9e3c-4fdaa833df53 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:21.363891 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cff584-63fe-4ed0-9e3c-4fdaa833df53 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:21.365767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cff584-63fe-4ed0-9e3c-4fdaa833df53 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-b6cff584-63fe-4ed0-9e3c-4fdaa833df53 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:21.375718 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:21.376767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:21.376767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:21.376767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:21.376767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:21.376767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:21.376767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:21.376767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:21.376767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:21.376767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:21.376767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:21.376767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:21.376767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:21.376767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:21.376767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:21.376767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:21.376767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:21.378004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:21.378004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:21.378004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:21.378004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:21.378004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:21.378004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:21.378004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:21.378004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:21.378004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:21.378004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:21.378004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:21.378004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:21.378004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 14:27:21.378004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:21.378004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 347/696] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Oct 12 14:27:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:21.384299 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac939cd-d1d1-4f3f-a428-0c4b51cd5d49 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:21.421880 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3e02a-3f08-4b75-9ca7-4364670e0bf7 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:21.422230 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3e02a-3f08-4b75-9ca7-4364670e0bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:21.422429 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3e02a-3f08-4b75-9ca7-4364670e0bf7 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:21.423822 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 348/697] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:21.430153 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5cd68-4233-4cff-982f-9874ec4ac16d None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:21.430438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5cd68-4233-4cff-982f-9874ec4ac16d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:21.430710 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5cd68-4233-4cff-982f-9874ec4ac16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:21.453451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-13d5cd68-4233-4cff-982f-9874ec4ac16d None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:21.454688 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-13d5cd68-4233-4cff-982f-9874ec4ac16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:21.458673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:21.513623 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac939cd-d1d1-4f3f-a428-0c4b51cd5d49 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:21.535279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 349/698] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:21.552966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03841a57-bce3-43d1-bd0c-d485b4926062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:21.553666 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03841a57-bce3-43d1-bd0c-d485b4926062 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:21.553944 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03841a57-bce3-43d1-bd0c-d485b4926062 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:21.554212 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03841a57-bce3-43d1-bd0c-d485b4926062 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:21.554860 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03841a57-bce3-43d1-bd0c-d485b4926062 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:21.555183 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03841a57-bce3-43d1-bd0c-d485b4926062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:21.556631 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03841a57-bce3-43d1-bd0c-d485b4926062 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:21.561863 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 350/699] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Oct 12 14:27:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:21.568236 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5cd68-4233-4cff-982f-9874ec4ac16d None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:21.675650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5cd68-4233-4cff-982f-9874ec4ac16d None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:21.711499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fe6bf-74b3-42c1-9ca4-b0ad77294ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:21.712354 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fe6bf-74b3-42c1-9ca4-b0ad77294ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:21.712774 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fe6bf-74b3-42c1-9ca4-b0ad77294ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:21.713196 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fe6bf-74b3-42c1-9ca4-b0ad77294ba0 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:21.713962 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fe6bf-74b3-42c1-9ca4-b0ad77294ba0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:21.714428 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fe6bf-74b3-42c1-9ca4-b0ad77294ba0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:21.716000 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fe6bf-74b3-42c1-9ca4-b0ad77294ba0 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:21.721454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 350/700] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Oct 12 14:27:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:21.728092 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03841a57-bce3-43d1-bd0c-d485b4926062 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:21.777610 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:21.829550 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fe6bf-74b3-42c1-9ca4-b0ad77294ba0 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:21.834165 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:21.835023 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03841a57-bce3-43d1-bd0c-d485b4926062 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:21.880571 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57459d2e-d85d-43fc-af09-ba633816edeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:21.881188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57459d2e-d85d-43fc-af09-ba633816edeb None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:21.881465 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57459d2e-d85d-43fc-af09-ba633816edeb None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:21.881738 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57459d2e-d85d-43fc-af09-ba633816edeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:21.882597 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57459d2e-d85d-43fc-af09-ba633816edeb None admin] RBAC: Adding query filter params (interface=public, service_id=4484e95c30cd42dfa87563891ea621c8, region_id=RegionOne) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:21.883322 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57459d2e-d85d-43fc-af09-ba633816edeb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:21.884964 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57459d2e-d85d-43fc-af09-ba633816edeb None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:21.894779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 351/701] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Oct 12 14:27:21 2022] GET /identity/v3/endpoints?service_id=4484e95c30cd42dfa87563891ea621c8&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:21.901266 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57459d2e-d85d-43fc-af09-ba633816edeb None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:21.945345 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fe6bf-74b3-42c1-9ca4-b0ad77294ba0 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:21.985733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcff38-768b-4457-828d-df49b616a119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:21.986368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcff38-768b-4457-828d-df49b616a119 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:21.986644 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcff38-768b-4457-828d-df49b616a119 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:21.986913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcff38-768b-4457-828d-df49b616a119 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:21.987517 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebcff38-768b-4457-828d-df49b616a119 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:21.989549 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebcff38-768b-4457-828d-df49b616a119 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-3ebcff38-768b-4457-828d-df49b616a119 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:22.000532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:22.001621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:22.001621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:22.001621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:22.001621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:22.001621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:22.001621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:22.001621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:22.001621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:22.001621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:22.001621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:22.001621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:22.001621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:22.001621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:22.001621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:22.001621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:22.001621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:22.002444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:22.002444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:22.002444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:22.002444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:22.002444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:22.002444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:22.002444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:22.002444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:22.002444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:22.002444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:22.002444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:22.002444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:22.002444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 14:27:22.002444 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:22.006634 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57459d2e-d85d-43fc-af09-ba633816edeb None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:22.007229 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 351/702] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Oct 12 14:27:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:22.008953 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcff38-768b-4457-828d-df49b616a119 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:22.016585 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:22.043357 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d304042-f0af-4c8c-9d39-3a9e0cc52028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:22.043971 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d304042-f0af-4c8c-9d39-3a9e0cc52028 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:22.044249 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d304042-f0af-4c8c-9d39-3a9e0cc52028 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:22.044524 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d304042-f0af-4c8c-9d39-3a9e0cc52028 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:22.045126 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d304042-f0af-4c8c-9d39-3a9e0cc52028 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:22.046720 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d304042-f0af-4c8c-9d39-3a9e0cc52028 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:22.059343 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 352/703] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Oct 12 14:27:21 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 14:27:22.087849 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:22.127512 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcff38-768b-4457-828d-df49b616a119 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:22.168926 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad549dc-0675-405b-b499-7edbc921cd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:22.169642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad549dc-0675-405b-b499-7edbc921cd53 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:22.169918 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad549dc-0675-405b-b499-7edbc921cd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:22.170187 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad549dc-0675-405b-b499-7edbc921cd53 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:22.170831 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad549dc-0675-405b-b499-7edbc921cd53 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:22.171129 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad549dc-0675-405b-b499-7edbc921cd53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:22.172534 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad549dc-0675-405b-b499-7edbc921cd53 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:22.178378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 352/704] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Oct 12 14:27:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:22.184143 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d304042-f0af-4c8c-9d39-3a9e0cc52028 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:22.295099 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d304042-f0af-4c8c-9d39-3a9e0cc52028 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:22.332102 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2a465-aebe-41a6-ab17-74f3e89e418e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:22.332726 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2a465-aebe-41a6-ab17-74f3e89e418e None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:22.333070 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2a465-aebe-41a6-ab17-74f3e89e418e None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:22.333540 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2a465-aebe-41a6-ab17-74f3e89e418e None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:22.334193 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2a465-aebe-41a6-ab17-74f3e89e418e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:22.334490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2a465-aebe-41a6-ab17-74f3e89e418e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:22.335901 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2a465-aebe-41a6-ab17-74f3e89e418e None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:22.341561 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 353/705] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Oct 12 14:27:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:22.347499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad549dc-0675-405b-b499-7edbc921cd53 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:22.451856 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad549dc-0675-405b-b499-7edbc921cd53 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:22.481858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:22.486892 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66afdb3-8e2c-4df3-ab2e-601bc02a2640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:22.487590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66afdb3-8e2c-4df3-ab2e-601bc02a2640 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:22.487868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66afdb3-8e2c-4df3-ab2e-601bc02a2640 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:22.488138 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66afdb3-8e2c-4df3-ab2e-601bc02a2640 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:22.488840 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66afdb3-8e2c-4df3-ab2e-601bc02a2640 None admin] RBAC: Adding query filter params (interface=public, service_id=5743606912914de88817e3233333d527, region_id=RegionOne) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:22.489217 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66afdb3-8e2c-4df3-ab2e-601bc02a2640 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:22.490587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66afdb3-8e2c-4df3-ab2e-601bc02a2640 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:22.496136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 353/706] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Oct 12 14:27:22 2022] GET /identity/v3/endpoints?service_id=5743606912914de88817e3233333d527&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:22.501798 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2a465-aebe-41a6-ab17-74f3e89e418e None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:22.543104 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:22.606545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2a465-aebe-41a6-ab17-74f3e89e418e None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:22.642353 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd267fd-bb0a-468b-986f-b169185fda7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:22.643289 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd267fd-bb0a-468b-986f-b169185fda7d None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:22.643709 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd267fd-bb0a-468b-986f-b169185fda7d None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:22.644120 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd267fd-bb0a-468b-986f-b169185fda7d None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:22.644879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd267fd-bb0a-468b-986f-b169185fda7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:22.647089 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd267fd-bb0a-468b-986f-b169185fda7d None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:22.653806 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 354/707] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Oct 12 14:27:22 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 14:27:23.461464 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67eda3-dec3-4845-82ba-1b31c1dd97e2 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:23.461987 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67eda3-dec3-4845-82ba-1b31c1dd97e2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:23.461987 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67eda3-dec3-4845-82ba-1b31c1dd97e2 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:23.463654 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 354/708] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:23.469113 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f9a46-9416-41c9-97a9-fc0bf9b4a202 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:23.469517 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f9a46-9416-41c9-97a9-fc0bf9b4a202 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:23.469656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f9a46-9416-41c9-97a9-fc0bf9b4a202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:23.487824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-3c7f9a46-9416-41c9-97a9-fc0bf9b4a202 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:23.489052 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-3c7f9a46-9416-41c9-97a9-fc0bf9b4a202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:23.563941 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 355/709] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:23.668253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701411f6-85f3-4be8-8713-b18eb1d15b2b None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:23.668630 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701411f6-85f3-4be8-8713-b18eb1d15b2b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:23.668979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701411f6-85f3-4be8-8713-b18eb1d15b2b None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:23.670664 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 355/710] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:23.675947 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d97a45-0926-4b00-997f-987c557b4ffc None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:23.676233 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d97a45-0926-4b00-997f-987c557b4ffc None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:23.676502 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d97a45-0926-4b00-997f-987c557b4ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:23.699684 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-a9d97a45-0926-4b00-997f-987c557b4ffc None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:23.700847 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-a9d97a45-0926-4b00-997f-987c557b4ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:23.726984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:23.769317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 356/711] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:23.844371 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701411f6-85f3-4be8-8713-b18eb1d15b2b None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:23.950967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701411f6-85f3-4be8-8713-b18eb1d15b2b None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:23.985999 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815e8b6-9338-4f5b-851f-a862a47b527f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:23.987050 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815e8b6-9338-4f5b-851f-a862a47b527f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:23.987569 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815e8b6-9338-4f5b-851f-a862a47b527f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:23.988085 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815e8b6-9338-4f5b-851f-a862a47b527f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:23.988878 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815e8b6-9338-4f5b-851f-a862a47b527f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:23.991238 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815e8b6-9338-4f5b-851f-a862a47b527f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-9815e8b6-9338-4f5b-851f-a862a47b527f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:24.000924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:24.001914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:24.001914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:24.001914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:24.001914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:24.001914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:24.001914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:24.001914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:24.001914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:24.001914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:24.001914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:24.001914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:24.001914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:24.001914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:24.001914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:24.001914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:24.001914 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:24.002744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:24.002744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:24.002744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:24.002744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:24.002744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:24.002744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:24.002744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:24.002744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:24.002744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:24.002744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:24.002744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:24.002744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:24.002744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 14:27:24.002744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:24.005598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 356/712] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:27:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:24.011640 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d97a45-0926-4b00-997f-987c557b4ffc None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:24.039640 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815e8b6-9338-4f5b-851f-a862a47b527f None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:24.103761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:24.131077 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d97a45-0926-4b00-997f-987c557b4ffc None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:24.150634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815e8b6-9338-4f5b-851f-a862a47b527f None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:24.175572 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7944f1b-e149-4926-8ffc-d837f5191a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:24.176736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7944f1b-e149-4926-8ffc-d837f5191a12 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:24.177716 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7944f1b-e149-4926-8ffc-d837f5191a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:24.178349 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7944f1b-e149-4926-8ffc-d837f5191a12 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:24.179575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7944f1b-e149-4926-8ffc-d837f5191a12 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:24.180321 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7944f1b-e149-4926-8ffc-d837f5191a12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:24.182058 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7944f1b-e149-4926-8ffc-d837f5191a12 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:24.194627 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 357/713] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Oct 12 14:27:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:24.194812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b869605-399d-41bb-b059-903744980194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:24.194812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b869605-399d-41bb-b059-903744980194 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:24.194812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b869605-399d-41bb-b059-903744980194 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:24.194812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b869605-399d-41bb-b059-903744980194 None admin] PATH_INFO: `/v3/services/image` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:24.195101 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b869605-399d-41bb-b059-903744980194 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:24.196512 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7944f1b-e149-4926-8ffc-d837f5191a12 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:24.197413 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b869605-399d-41bb-b059-903744980194 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-3b869605-399d-41bb-b059-903744980194 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:24.209313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:24.210315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:24.210315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:24.210315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:24.210315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:24.210315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:24.210315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:24.210315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:24.210315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:24.210315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:24.210315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:24.210315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:24.210315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:24.210315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:24.210315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:24.210315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:24.210315 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:24.211350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:24.211350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:24.211350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:24.211350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:24.211350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:24.211350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:24.211350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:24.211350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:24.211350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:24.211350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:24.211350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:24.211350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:24.211350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 14:27:24.211350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:24.211350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 357/714] 10.222.0.16 () {56 vars in 1105 bytes} [Wed Oct 12 14:27:24 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:24.215903 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b869605-399d-41bb-b059-903744980194 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:24.231709 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:24.322129 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7944f1b-e149-4926-8ffc-d837f5191a12 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:24.344958 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b869605-399d-41bb-b059-903744980194 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:24.358795 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c6289-ede9-4bd6-a326-683b2d4460ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:24.359742 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c6289-ede9-4bd6-a326-683b2d4460ae None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:24.360263 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c6289-ede9-4bd6-a326-683b2d4460ae None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:24.360730 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c6289-ede9-4bd6-a326-683b2d4460ae None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:24.361551 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c6289-ede9-4bd6-a326-683b2d4460ae None admin] RBAC: Adding query filter params (type=compute) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:24.362046 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c6289-ede9-4bd6-a326-683b2d4460ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:24.363636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c6289-ede9-4bd6-a326-683b2d4460ae None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:24.369497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:24.371571 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 358/715] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Oct 12 14:27:24 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:24.377177 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd91c0-8f51-4adf-88bf-4178c18a752c None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:24.377455 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd91c0-8f51-4adf-88bf-4178c18a752c None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:24.377722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd91c0-8f51-4adf-88bf-4178c18a752c None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:24.379303 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 359/716] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:27:24.386943 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5cc55-f88f-4bb8-b369-c58a0805e5fa None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:24.387264 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5cc55-f88f-4bb8-b369-c58a0805e5fa None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:24.387540 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5cc55-f88f-4bb8-b369-c58a0805e5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:24.394395 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663102ed-ff5e-48db-8a53-62692600875a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:24.395365 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663102ed-ff5e-48db-8a53-62692600875a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:24.395792 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663102ed-ff5e-48db-8a53-62692600875a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:24.396210 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663102ed-ff5e-48db-8a53-62692600875a None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:24.396866 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663102ed-ff5e-48db-8a53-62692600875a None admin] RBAC: Adding query filter params (name=image) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:24.397168 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663102ed-ff5e-48db-8a53-62692600875a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:24.398633 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663102ed-ff5e-48db-8a53-62692600875a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:24.405055 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 358/717] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:24.409531 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-b1a5cc55-f88f-4bb8-b369-c58a0805e5fa None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:24.410843 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-b1a5cc55-f88f-4bb8-b369-c58a0805e5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:24.416510 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663102ed-ff5e-48db-8a53-62692600875a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:24.520184 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 360/718] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 14:27:24.554844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663102ed-ff5e-48db-8a53-62692600875a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:24.605092 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a2ed2-d581-421d-bacb-1c6742869a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:24.605707 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a2ed2-d581-421d-bacb-1c6742869a34 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:24.605982 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a2ed2-d581-421d-bacb-1c6742869a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:24.606254 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a2ed2-d581-421d-bacb-1c6742869a34 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:24.606879 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a2ed2-d581-421d-bacb-1c6742869a34 None admin] RBAC: Adding query filter params (type=image) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:24.607272 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a2ed2-d581-421d-bacb-1c6742869a34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:24.608628 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a2ed2-d581-421d-bacb-1c6742869a34 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:24.613447 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 359/719] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:24 2022] GET /identity/v3/services?type=image => generated 122 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:24.798230 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5cc55-f88f-4bb8-b369-c58a0805e5fa None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:24.821457 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:24.903311 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5cc55-f88f-4bb8-b369-c58a0805e5fa None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:24.904571 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891053c-e72f-475f-a8db-4de16a623365 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:24.904968 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891053c-e72f-475f-a8db-4de16a623365 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:24.905248 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891053c-e72f-475f-a8db-4de16a623365 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:24.906937 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 360/720] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:27:24.912087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acbabb-e5ee-45cf-9b47-a9e196a3e391 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:24.912353 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acbabb-e5ee-45cf-9b47-a9e196a3e391 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:24.912625 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acbabb-e5ee-45cf-9b47-a9e196a3e391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:24.935495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-f1acbabb-e5ee-45cf-9b47-a9e196a3e391 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:24.937147 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f1acbabb-e5ee-45cf-9b47-a9e196a3e391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:24.939256 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce8733-9f26-4d88-aaf8-c3d4807d9764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:24.939893 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce8733-9f26-4d88-aaf8-c3d4807d9764 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:24.940166 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce8733-9f26-4d88-aaf8-c3d4807d9764 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:24.940432 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce8733-9f26-4d88-aaf8-c3d4807d9764 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:24.941006 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce8733-9f26-4d88-aaf8-c3d4807d9764 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:24.942545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce8733-9f26-4d88-aaf8-c3d4807d9764 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-d9ce8733-9f26-4d88-aaf8-c3d4807d9764 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:24.952079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:24.953081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:24.953081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:24.953081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:24.953081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:24.953081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:24.953081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:24.953081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:24.953081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:24.953081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:24.953081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:24.953081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:24.953081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:24.953081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:24.953081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:24.953081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:24.953081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:24.953883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:24.953883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:24.953883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:24.953883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:24.953883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:24.953883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:24.953883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:24.953883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:24.953883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:24.953883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:24.953883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:24.953883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:24.953883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 14:27:24.953883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:24.953883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 361/721] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Oct 12 14:27:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:24.961284 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce8733-9f26-4d88-aaf8-c3d4807d9764 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:24.994711 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:25.017500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 361/722] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 14:27:25.049936 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:25.066713 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce8733-9f26-4d88-aaf8-c3d4807d9764 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:25.102039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdb956-831c-4344-995f-7fefc7ea3722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:25.102705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdb956-831c-4344-995f-7fefc7ea3722 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:25.103280 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdb956-831c-4344-995f-7fefc7ea3722 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:25.103553 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdb956-831c-4344-995f-7fefc7ea3722 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:25.104189 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdb956-831c-4344-995f-7fefc7ea3722 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:25.104525 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdb956-831c-4344-995f-7fefc7ea3722 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:25.105984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdb956-831c-4344-995f-7fefc7ea3722 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:25.111489 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 362/723] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Oct 12 14:27:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:25.117293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acbabb-e5ee-45cf-9b47-a9e196a3e391 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:25.219756 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acbabb-e5ee-45cf-9b47-a9e196a3e391 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:25.258908 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607f868-f91d-420b-be9e-92498dcb1e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:25.259547 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607f868-f91d-420b-be9e-92498dcb1e30 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:25.259817 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607f868-f91d-420b-be9e-92498dcb1e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:25.260081 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607f868-f91d-420b-be9e-92498dcb1e30 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:25.260708 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607f868-f91d-420b-be9e-92498dcb1e30 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:25.261003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607f868-f91d-420b-be9e-92498dcb1e30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:25.262343 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607f868-f91d-420b-be9e-92498dcb1e30 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:25.267480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 362/724] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Oct 12 14:27:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:25.276480 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdb956-831c-4344-995f-7fefc7ea3722 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:25.311699 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607f868-f91d-420b-be9e-92498dcb1e30 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:25.369944 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:25.401661 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdb956-831c-4344-995f-7fefc7ea3722 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:25.443412 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607f868-f91d-420b-be9e-92498dcb1e30 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:25.451398 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651cd12d-24ed-462e-8e67-18ad5d786d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:25.452339 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cd12d-24ed-462e-8e67-18ad5d786d6b None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:25.452819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cd12d-24ed-462e-8e67-18ad5d786d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:25.453289 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cd12d-24ed-462e-8e67-18ad5d786d6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:25.454153 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651cd12d-24ed-462e-8e67-18ad5d786d6b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:25.455825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651cd12d-24ed-462e-8e67-18ad5d786d6b None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:25.486537 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 363/725] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Oct 12 14:27:25 2022] POST /identity/v3/endpoints => generated 349 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:27:25.492461 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fdc9e-f938-413c-8fff-cb6902824f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:25.493113 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fdc9e-f938-413c-8fff-cb6902824f79 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:25.493387 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fdc9e-f938-413c-8fff-cb6902824f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:25.493654 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fdc9e-f938-413c-8fff-cb6902824f79 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:25.494246 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639fdc9e-f938-413c-8fff-cb6902824f79 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:25.495705 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639fdc9e-f938-413c-8fff-cb6902824f79 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-639fdc9e-f938-413c-8fff-cb6902824f79 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:25.505859 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:25.506897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:25.506897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:25.506897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:25.506897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:25.506897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:25.506897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:25.506897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:25.506897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:25.506897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:25.506897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:25.506897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:25.506897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:25.506897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:25.506897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:25.506897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:25.506897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:25.507744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:25.507744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:25.507744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:25.507744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:25.507744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:25.507744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:25.507744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:25.507744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:25.507744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:25.507744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:25.507744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:25.507744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:25.507744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 14:27:25.507744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:25.511475 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 363/726] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Oct 12 14:27:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:25.513584 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651cd12d-24ed-462e-8e67-18ad5d786d6b None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:25.518506 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:25.639330 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651cd12d-24ed-462e-8e67-18ad5d786d6b None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:25.682610 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966c491-ffbc-4b5c-a258-306cd9b00297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:25.683404 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966c491-ffbc-4b5c-a258-306cd9b00297 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:25.683680 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966c491-ffbc-4b5c-a258-306cd9b00297 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:25.683948 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966c491-ffbc-4b5c-a258-306cd9b00297 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:25.684582 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966c491-ffbc-4b5c-a258-306cd9b00297 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:25.684879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966c491-ffbc-4b5c-a258-306cd9b00297 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:25.687018 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966c491-ffbc-4b5c-a258-306cd9b00297 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:25.691941 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 364/727] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Oct 12 14:27:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:25.697798 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fdc9e-f938-413c-8fff-cb6902824f79 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:25.801642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fdc9e-f938-413c-8fff-cb6902824f79 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:25.809255 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:25.838562 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe4c49-517b-4dec-b09f-9d77e62e6312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:25.839199 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe4c49-517b-4dec-b09f-9d77e62e6312 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:25.839482 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe4c49-517b-4dec-b09f-9d77e62e6312 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:25.839756 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe4c49-517b-4dec-b09f-9d77e62e6312 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:25.840382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe4c49-517b-4dec-b09f-9d77e62e6312 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:25.840677 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe4c49-517b-4dec-b09f-9d77e62e6312 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:25.842030 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe4c49-517b-4dec-b09f-9d77e62e6312 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:25.851415 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 364/728] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Oct 12 14:27:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:25.853284 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966c491-ffbc-4b5c-a258-306cd9b00297 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:25.923008 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:25.960711 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966c491-ffbc-4b5c-a258-306cd9b00297 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:25.997189 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeaa364-06a6-4cd3-9c53-f84d594be1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:25.997797 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaa364-06a6-4cd3-9c53-f84d594be1ec None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:25.998126 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaa364-06a6-4cd3-9c53-f84d594be1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:25.998447 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaa364-06a6-4cd3-9c53-f84d594be1ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:25.999172 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeaa364-06a6-4cd3-9c53-f84d594be1ec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:26.000616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeaa364-06a6-4cd3-9c53-f84d594be1ec None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:26.022640 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 365/729] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Oct 12 14:27:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:27:26.632877 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0412a7a-669b-4dd3-90a0-6c44fd112035 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:26.633158 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0412a7a-669b-4dd3-90a0-6c44fd112035 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:26.633428 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0412a7a-669b-4dd3-90a0-6c44fd112035 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:26.634846 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 365/730] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:26.640446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821181b-5bc5-4205-a7b6-9aa30415dfd7 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:26.640906 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821181b-5bc5-4205-a7b6-9aa30415dfd7 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:26.641030 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821181b-5bc5-4205-a7b6-9aa30415dfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:26.660851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-f821181b-5bc5-4205-a7b6-9aa30415dfd7 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:26.662607 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-f821181b-5bc5-4205-a7b6-9aa30415dfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:26.741876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 366/731] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:26.920031 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ec9cf-2a0f-4795-b8a3-702ab5c3f876 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:26.921039 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ec9cf-2a0f-4795-b8a3-702ab5c3f876 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:26.921536 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ec9cf-2a0f-4795-b8a3-702ab5c3f876 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:26.923335 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 366/732] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:26.929020 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f44df-e548-4f82-9613-c7eddcc940a3 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:26.929535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f44df-e548-4f82-9613-c7eddcc940a3 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:26.930006 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f44df-e548-4f82-9613-c7eddcc940a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:26.949358 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-a00f44df-e548-4f82-9613-c7eddcc940a3 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:26.950918 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-a00f44df-e548-4f82-9613-c7eddcc940a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:27.002128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:27.011955 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ec9cf-2a0f-4795-b8a3-702ab5c3f876 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:27.027922 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 367/733] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:27.121456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ec9cf-2a0f-4795-b8a3-702ab5c3f876 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:27.156030 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d528a5b4-0d2d-430b-b4ea-4d7360b49753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:27.156632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528a5b4-0d2d-430b-b4ea-4d7360b49753 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:27.156901 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528a5b4-0d2d-430b-b4ea-4d7360b49753 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:27.157166 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528a5b4-0d2d-430b-b4ea-4d7360b49753 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:27.157815 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d528a5b4-0d2d-430b-b4ea-4d7360b49753 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:27.159178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d528a5b4-0d2d-430b-b4ea-4d7360b49753 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:27.178289 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 367/734] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Oct 12 14:27:27 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 14:27:27.313955 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f44df-e548-4f82-9613-c7eddcc940a3 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:27.422073 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f44df-e548-4f82-9613-c7eddcc940a3 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:27.465304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe17eb3-8058-4850-a657-1ae77b0524c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:27.466216 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe17eb3-8058-4850-a657-1ae77b0524c8 None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:27.466646 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe17eb3-8058-4850-a657-1ae77b0524c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:27.467061 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe17eb3-8058-4850-a657-1ae77b0524c8 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:27.467868 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe17eb3-8058-4850-a657-1ae77b0524c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:27.469379 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe17eb3-8058-4850-a657-1ae77b0524c8 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:27.488303 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 368/735] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Oct 12 14:27:27 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:27:27.860829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b413733-c666-40ea-b6e7-0b37c62dbe14 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:27.861244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b413733-c666-40ea-b6e7-0b37c62dbe14 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:27.861669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b413733-c666-40ea-b6e7-0b37c62dbe14 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:27.863374 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 368/736] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:27.869293 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644120b-8c3e-4c06-a078-28b581d74437 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:27.869764 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644120b-8c3e-4c06-a078-28b581d74437 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:27.870040 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644120b-8c3e-4c06-a078-28b581d74437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:27.889509 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-d644120b-8c3e-4c06-a078-28b581d74437 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:27.890681 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-d644120b-8c3e-4c06-a078-28b581d74437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:27.921719 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:27.973317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 369/737] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:27 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:28.246785 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b413733-c666-40ea-b6e7-0b37c62dbe14 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:28.260821 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799448e-1440-48e6-a6a0-f3e83f238899 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:28.261127 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799448e-1440-48e6-a6a0-f3e83f238899 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:28.261360 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799448e-1440-48e6-a6a0-f3e83f238899 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:28.262946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 370/738] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:27:28.267809 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad29cae-e32d-4162-99e2-4c899a159fc7 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:28.268086 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad29cae-e32d-4162-99e2-4c899a159fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:28.268359 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad29cae-e32d-4162-99e2-4c899a159fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:28.281151 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:28.290944 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-5ad29cae-e32d-4162-99e2-4c899a159fc7 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:28.292150 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-5ad29cae-e32d-4162-99e2-4c899a159fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:28.369789 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b413733-c666-40ea-b6e7-0b37c62dbe14 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:28.383300 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 371/739] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:28 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 14:27:28.413514 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe330e2b-e3ae-4cc8-861d-842792c07c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:28.414158 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe330e2b-e3ae-4cc8-861d-842792c07c70 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:28.414476 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe330e2b-e3ae-4cc8-861d-842792c07c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:28.414847 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe330e2b-e3ae-4cc8-861d-842792c07c70 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:28.415483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe330e2b-e3ae-4cc8-861d-842792c07c70 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:28.416870 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe330e2b-e3ae-4cc8-861d-842792c07c70 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-fe330e2b-e3ae-4cc8-861d-842792c07c70 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:28.426970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:28.428295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:28.428295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:28.428295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:28.428295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:28.428295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:28.428295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:28.428295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:28.428295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:28.428295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:28.428295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:28.428295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:28.428295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:28.428295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:28.428295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:28.428295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:28.428295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:28.429120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:28.429120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:28.429120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:28.429120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:28.429120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:28.429120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:28.429120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:28.429120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:28.429120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:28.429120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:28.429120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:28.429120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:28.429120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 14:27:28.429120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:28.429120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 369/740] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Oct 12 14:27:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:28.438492 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad29cae-e32d-4162-99e2-4c899a159fc7 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:28.537508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:28.547329 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad29cae-e32d-4162-99e2-4c899a159fc7 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:28.588027 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bbf6d-741b-42b7-97c5-a63f4e298ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:28.588641 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bbf6d-741b-42b7-97c5-a63f4e298ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:28.588917 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bbf6d-741b-42b7-97c5-a63f4e298ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:28.589186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bbf6d-741b-42b7-97c5-a63f4e298ec7 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:28.589811 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bbf6d-741b-42b7-97c5-a63f4e298ec7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:28.590111 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bbf6d-741b-42b7-97c5-a63f4e298ec7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:28.591502 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bbf6d-741b-42b7-97c5-a63f4e298ec7 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:28.597028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 372/741] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Oct 12 14:27:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:28.602886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe330e2b-e3ae-4cc8-861d-842792c07c70 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:28.658057 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bbf6d-741b-42b7-97c5-a63f4e298ec7 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:28.716221 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:28.726698 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe330e2b-e3ae-4cc8-861d-842792c07c70 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:28.769472 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bbf6d-741b-42b7-97c5-a63f4e298ec7 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:28.773269 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ccf31-6353-46dd-9428-ee6f80c1dbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:28.773957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ccf31-6353-46dd-9428-ee6f80c1dbe0 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:28.774299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ccf31-6353-46dd-9428-ee6f80c1dbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:28.774901 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ccf31-6353-46dd-9428-ee6f80c1dbe0 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:28.775652 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ccf31-6353-46dd-9428-ee6f80c1dbe0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:28.776033 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ccf31-6353-46dd-9428-ee6f80c1dbe0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:28.777456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ccf31-6353-46dd-9428-ee6f80c1dbe0 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:28.787243 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 370/742] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Oct 12 14:27:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:28.791357 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ccf31-6353-46dd-9428-ee6f80c1dbe0 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:28.817081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18628fb1-d1a8-4d62-b42c-c617ce12c513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:28.818018 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18628fb1-d1a8-4d62-b42c-c617ce12c513 None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:28.818297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18628fb1-d1a8-4d62-b42c-c617ce12c513 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:28.818659 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18628fb1-d1a8-4d62-b42c-c617ce12c513 None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:28.819496 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18628fb1-d1a8-4d62-b42c-c617ce12c513 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:28.821620 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18628fb1-d1a8-4d62-b42c-c617ce12c513 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:28.825078 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: WARNING keystone.server.flask.common [None req-18628fb1-d1a8-4d62-b42c-c617ce12c513 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 12 14:27:28.848222 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.sql.core [None req-18628fb1-d1a8-4d62-b42c-c617ce12c513 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 12 14:27:28.848222 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 14:27:28.848222 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [parameters: {'id': 'c2a1714205fc44eab89ec174af75b92e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 12 14:27:28.848222 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81566) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 14:27:28.871028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-18628fb1-d1a8-4d62-b42c-c617ce12c513 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c2a1714205fc44eab89ec174af75b92e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c2a1714205fc44eab89ec174af75b92e. Oct 12 14:27:28.871028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:28.871028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 14:27:28.871028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 14:27:28.871028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 14:27:28.871028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 14:27:28.871028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 14:27:28.871028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 14:27:28.871028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 14:27:28.871028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application conn.query(q) Oct 12 14:27:28.871028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 14:27:28.871028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 14:27:28.871028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application result.read() Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 12 14:27:28.872895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application next(self.gen) Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application yield resource Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application next(self.gen) Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise self.value Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application session.commit() Oct 12 14:27:28.873818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self.session.flush() Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application compat.raise_( Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 14:27:28.874705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 14:27:28.876642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 14:27:28.876642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 14:27:28.876642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 14:27:28.876642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 14:27:28.876642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 12 14:27:28.876642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 12 14:27:28.876642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 14:27:28.876642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 14:27:28.876642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 14:27:28.876642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 14:27:28.876642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 14:27:28.876642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 14:27:28.876642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 14:27:28.876642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 14:27:28.876642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application conn.query(q) Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application result.read() Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 14:27:28.878431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 14:27:28.888362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 14:27:28.888362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 14:27:28.888362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 14:27:28.888362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 14:27:28.888362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 14:27:28.888362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 14:27:28.888362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 12 14:27:28.888362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 14:27:28.888362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [parameters: {'id': 'c2a1714205fc44eab89ec174af75b92e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 12 14:27:28.888362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 14:27:28.888362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:28.888362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:28.888362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:28.888362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:28.888362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:28.890017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:28.891779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:28.891779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 12 14:27:28.891779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 12 14:27:28.891779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:28.891779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:28.891779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 12 14:27:28.891779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 12 14:27:28.891779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c2a1714205fc44eab89ec174af75b92e. Oct 12 14:27:28.891779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:28.891779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 373/743] 10.222.0.16 () {60 vars in 1140 bytes} [Wed Oct 12 14:27:28 2022] POST /identity/v3/projects => generated 262 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:28.891779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18628fb1-d1a8-4d62-b42c-c617ce12c513 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:28.901953 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:28.912984 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ccf31-6353-46dd-9428-ee6f80c1dbe0 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:28.961068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c448e-8430-4601-9f44-197008920163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:28.961624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c448e-8430-4601-9f44-197008920163 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:28.961906 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c448e-8430-4601-9f44-197008920163 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:28.962178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c448e-8430-4601-9f44-197008920163 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:28.962896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c448e-8430-4601-9f44-197008920163 None admin] RBAC: Adding query filter params (interface=admin, service_id=4484e95c30cd42dfa87563891ea621c8, region_id=RegionOne) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:28.963205 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c448e-8430-4601-9f44-197008920163 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:28.964571 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c448e-8430-4601-9f44-197008920163 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:28.976459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 371/744] 10.222.0.16 () {56 vars in 1249 bytes} [Wed Oct 12 14:27:28 2022] GET /identity/v3/endpoints?service_id=4484e95c30cd42dfa87563891ea621c8&interface=admin®ion_id=RegionOne => generated 193 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:28.977985 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c448e-8430-4601-9f44-197008920163 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:29.003116 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18628fb1-d1a8-4d62-b42c-c617ce12c513 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:29.006961 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:29.041481 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8ac00-d2ee-450f-b7f9-487bf2235b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:29.042428 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8ac00-d2ee-450f-b7f9-487bf2235b76 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:29.042932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8ac00-d2ee-450f-b7f9-487bf2235b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:29.043492 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8ac00-d2ee-450f-b7f9-487bf2235b76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:29.050735 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8ac00-d2ee-450f-b7f9-487bf2235b76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:29.052740 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8ac00-d2ee-450f-b7f9-487bf2235b76 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:29.064588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-7fc8ac00-d2ee-450f-b7f9-487bf2235b76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:29.064588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:29.064588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:29.064588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:29.064588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:29.064588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:29.064588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:29.064588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:29.064588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:29.064588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:29.064588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:29.064588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:29.064588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:27:29.064588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:27:29.064588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:29.065490 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:29.066338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:27:29.066338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:27:29.066338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:27:29.066338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:27:29.066338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:27:29.066338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:27:29.066338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:29.066338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:29.073884 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 374/745] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:27:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:29.074981 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8ac00-d2ee-450f-b7f9-487bf2235b76 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:29.095098 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c448e-8430-4601-9f44-197008920163 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:29.137204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b8349-c0ae-4899-93b8-8fcb1184fc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:29.137968 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b8349-c0ae-4899-93b8-8fcb1184fc97 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:29.138366 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b8349-c0ae-4899-93b8-8fcb1184fc97 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:29.138648 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b8349-c0ae-4899-93b8-8fcb1184fc97 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:29.139279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b8349-c0ae-4899-93b8-8fcb1184fc97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:29.141098 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b8349-c0ae-4899-93b8-8fcb1184fc97 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:29.149496 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 372/746] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Oct 12 14:27:28 2022] GET /identity/v3/services => generated 1957 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 14:27:29.160744 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:29.199032 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8ac00-d2ee-450f-b7f9-487bf2235b76 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:29.238952 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9711744-2339-470e-875d-5b9adaef2c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:29.240233 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9711744-2339-470e-875d-5b9adaef2c59 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:29.240795 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9711744-2339-470e-875d-5b9adaef2c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:29.241254 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9711744-2339-470e-875d-5b9adaef2c59 None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:29.242150 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9711744-2339-470e-875d-5b9adaef2c59 None admin] RBAC: Adding query filter params (name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:29.242741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9711744-2339-470e-875d-5b9adaef2c59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:29.244474 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9711744-2339-470e-875d-5b9adaef2c59 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:29.267638 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 375/747] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Oct 12 14:27:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:29.486844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2bc23-a54c-4ed1-a1fc-40c3adf7a460 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:29.487381 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2bc23-a54c-4ed1-a1fc-40c3adf7a460 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:29.487487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2bc23-a54c-4ed1-a1fc-40c3adf7a460 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:29.489066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 373/748] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:29.494513 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e0b15-3b8c-48b0-b96b-c4f0f4670a5d None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:29.494989 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e0b15-3b8c-48b0-b96b-c4f0f4670a5d None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:29.495434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e0b15-3b8c-48b0-b96b-c4f0f4670a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:29.515537 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-a16e0b15-3b8c-48b0-b96b-c4f0f4670a5d None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:29.517027 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-a16e0b15-3b8c-48b0-b96b-c4f0f4670a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:29.590308 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 376/749] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:29 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:29.762446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb39627-82fc-4f71-8b80-8a43a17701ee None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:29.762740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb39627-82fc-4f71-8b80-8a43a17701ee None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:29.762924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb39627-82fc-4f71-8b80-8a43a17701ee None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:29.764516 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 374/750] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:29.769671 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d34e7-d43c-465e-a49d-acd90e56f25f None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:29.769952 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d34e7-d43c-465e-a49d-acd90e56f25f None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:29.770218 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d34e7-d43c-465e-a49d-acd90e56f25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:29.789495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-c84d34e7-d43c-465e-a49d-acd90e56f25f None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:29.790669 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-c84d34e7-d43c-465e-a49d-acd90e56f25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:29.850641 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:29.862840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 377/751] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:29 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:29.870518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb39627-82fc-4f71-8b80-8a43a17701ee None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:29.975616 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:29.980279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb39627-82fc-4f71-8b80-8a43a17701ee None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:30.015675 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01a849-43f9-4535-8f32-c21beb7b2351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:30.016242 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01a849-43f9-4535-8f32-c21beb7b2351 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:30.016567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01a849-43f9-4535-8f32-c21beb7b2351 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:30.016845 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01a849-43f9-4535-8f32-c21beb7b2351 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:30.017435 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01a849-43f9-4535-8f32-c21beb7b2351 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:30.018792 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01a849-43f9-4535-8f32-c21beb7b2351 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-0e01a849-43f9-4535-8f32-c21beb7b2351 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:30.028913 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:30.029973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:30.029973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:30.029973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:30.029973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:30.029973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:30.029973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:30.029973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:30.029973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:30.029973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:30.029973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:30.029973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:30.029973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:30.029973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:30.029973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:30.029973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:30.029973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:30.030801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:30.030801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:30.030801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:30.030801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:30.030801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:30.030801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:30.030801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:30.030801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:30.030801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:30.030801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:30.030801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:30.030801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:30.030801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 14:27:30.030801 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:30.033721 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 375/752] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:27:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:30.036400 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d34e7-d43c-465e-a49d-acd90e56f25f None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:30.138014 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01a849-43f9-4535-8f32-c21beb7b2351 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:30.147057 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d34e7-d43c-465e-a49d-acd90e56f25f None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:30.184325 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b26c0-a81a-43b1-a8e5-564ff1d5e389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:30.185295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b26c0-a81a-43b1-a8e5-564ff1d5e389 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:30.185812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b26c0-a81a-43b1-a8e5-564ff1d5e389 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:30.186279 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b26c0-a81a-43b1-a8e5-564ff1d5e389 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:30.187111 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b26c0-a81a-43b1-a8e5-564ff1d5e389 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:30.187710 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b26c0-a81a-43b1-a8e5-564ff1d5e389 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:30.189466 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b26c0-a81a-43b1-a8e5-564ff1d5e389 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:30.195657 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 378/753] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Oct 12 14:27:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:30.201915 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b26c0-a81a-43b1-a8e5-564ff1d5e389 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:30.254009 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01a849-43f9-4535-8f32-c21beb7b2351 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:30.275307 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:30.290367 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955a009-7828-430d-8d52-c1c056764018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:30.291457 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955a009-7828-430d-8d52-c1c056764018 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:30.291802 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955a009-7828-430d-8d52-c1c056764018 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:30.292143 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955a009-7828-430d-8d52-c1c056764018 None admin] PATH_INFO: `/v3/services/image` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:30.292810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955a009-7828-430d-8d52-c1c056764018 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:30.294279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955a009-7828-430d-8d52-c1c056764018 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-3955a009-7828-430d-8d52-c1c056764018 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:30.305378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:30.306391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:30.306391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:30.306391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:30.306391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:30.306391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:30.306391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:30.306391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:30.306391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:30.306391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:30.306391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:30.306391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:30.306391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:30.306391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:30.306391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:30.306391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:30.306391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:30.307567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:30.307567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:30.307567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:30.307567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:30.307567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:30.307567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:30.307567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:30.307567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:30.307567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:30.307567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:30.307567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:30.307567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:30.307567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 14:27:30.307567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:30.311486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 376/754] 10.222.0.16 () {56 vars in 1105 bytes} [Wed Oct 12 14:27:30 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:30.312398 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955a009-7828-430d-8d52-c1c056764018 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:30.340026 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b26c0-a81a-43b1-a8e5-564ff1d5e389 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:30.343544 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:30.389956 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb6470-0e7b-4d26-87a1-ad2363a04f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:30.390681 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb6470-0e7b-4d26-87a1-ad2363a04f6c None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:30.391030 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb6470-0e7b-4d26-87a1-ad2363a04f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:30.391394 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb6470-0e7b-4d26-87a1-ad2363a04f6c None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:30.392532 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb6470-0e7b-4d26-87a1-ad2363a04f6c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:30.392909 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb6470-0e7b-4d26-87a1-ad2363a04f6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:30.394718 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb6470-0e7b-4d26-87a1-ad2363a04f6c None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:30.402286 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 379/755] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Oct 12 14:27:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:30.408178 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb6470-0e7b-4d26-87a1-ad2363a04f6c None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:30.422808 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955a009-7828-430d-8d52-c1c056764018 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:30.460689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81bb32-0c4c-4a37-ac30-5ca1e3453f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:30.461386 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81bb32-0c4c-4a37-ac30-5ca1e3453f4c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:30.461750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81bb32-0c4c-4a37-ac30-5ca1e3453f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:30.462090 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81bb32-0c4c-4a37-ac30-5ca1e3453f4c None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:30.462844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81bb32-0c4c-4a37-ac30-5ca1e3453f4c None admin] RBAC: Adding query filter params (name=image) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:30.463239 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81bb32-0c4c-4a37-ac30-5ca1e3453f4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:30.464778 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81bb32-0c4c-4a37-ac30-5ca1e3453f4c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:30.475402 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 377/756] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:30.476391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81bb32-0c4c-4a37-ac30-5ca1e3453f4c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:30.528064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb6470-0e7b-4d26-87a1-ad2363a04f6c None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:30.551178 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:30.583480 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6990ba-2e7c-41a6-a4ca-fc819bbc8598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:30.584951 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6990ba-2e7c-41a6-a4ca-fc819bbc8598 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:30.585986 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:30.587426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6990ba-2e7c-41a6-a4ca-fc819bbc8598 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:30.587612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6990ba-2e7c-41a6-a4ca-fc819bbc8598 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:30.588533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6990ba-2e7c-41a6-a4ca-fc819bbc8598 None admin] RBAC: Adding query filter params (interface=public, service_id=7cc39046ebd4481fa3edf98e99f4c136, region_id=RegionOne) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:30.588920 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6990ba-2e7c-41a6-a4ca-fc819bbc8598 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:30.590372 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6990ba-2e7c-41a6-a4ca-fc819bbc8598 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:30.597345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 380/757] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Oct 12 14:27:30 2022] GET /identity/v3/endpoints?service_id=7cc39046ebd4481fa3edf98e99f4c136&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:30.604581 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6990ba-2e7c-41a6-a4ca-fc819bbc8598 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:30.616351 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81bb32-0c4c-4a37-ac30-5ca1e3453f4c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:30.657370 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc04acf-6e9f-49d1-9b69-754a7a9f13df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:30.658129 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc04acf-6e9f-49d1-9b69-754a7a9f13df None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:30.658501 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc04acf-6e9f-49d1-9b69-754a7a9f13df None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:30.658903 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc04acf-6e9f-49d1-9b69-754a7a9f13df None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:30.659669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc04acf-6e9f-49d1-9b69-754a7a9f13df None admin] RBAC: Adding query filter params (type=image) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:30.660099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc04acf-6e9f-49d1-9b69-754a7a9f13df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:30.661545 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc04acf-6e9f-49d1-9b69-754a7a9f13df None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:30.672400 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 378/758] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:30.673484 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc04acf-6e9f-49d1-9b69-754a7a9f13df None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:30.732651 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6990ba-2e7c-41a6-a4ca-fc819bbc8598 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:30.768676 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da1bd2-babd-4254-8594-f3fd805507fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:30.769534 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da1bd2-babd-4254-8594-f3fd805507fd None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:30.769785 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da1bd2-babd-4254-8594-f3fd805507fd None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:30.770149 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da1bd2-babd-4254-8594-f3fd805507fd None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:30.770840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da1bd2-babd-4254-8594-f3fd805507fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:30.772858 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da1bd2-babd-4254-8594-f3fd805507fd None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:30.787575 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 381/759] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Oct 12 14:27:30 2022] GET /identity/v3/services => generated 1957 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 14:27:30.795668 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc04acf-6e9f-49d1-9b69-754a7a9f13df None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:30.841124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab340663-a74b-4d32-823d-79be550c7065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:30.842048 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab340663-a74b-4d32-823d-79be550c7065 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:30.842475 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab340663-a74b-4d32-823d-79be550c7065 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:30.842887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab340663-a74b-4d32-823d-79be550c7065 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:30.843794 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab340663-a74b-4d32-823d-79be550c7065 None admin] RBAC: Adding query filter params (interface=public, service_id=7d7d8b0562a540c39434b280f7452ae6, region_id=RegionOne) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:30.844272 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab340663-a74b-4d32-823d-79be550c7065 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:30.845852 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab340663-a74b-4d32-823d-79be550c7065 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:30.853172 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 379/760] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Oct 12 14:27:30 2022] GET /identity/v3/endpoints?service_id=7d7d8b0562a540c39434b280f7452ae6&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:30.860518 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da1bd2-babd-4254-8594-f3fd805507fd None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:30.920929 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:30.979910 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da1bd2-babd-4254-8594-f3fd805507fd None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:31.015768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588381c6-341c-441a-81f6-cc5e4c0825f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:31.016622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588381c6-341c-441a-81f6-cc5e4c0825f9 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:31.017066 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588381c6-341c-441a-81f6-cc5e4c0825f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:31.017482 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588381c6-341c-441a-81f6-cc5e4c0825f9 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:31.018229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588381c6-341c-441a-81f6-cc5e4c0825f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:31.019778 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588381c6-341c-441a-81f6-cc5e4c0825f9 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:31.027764 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 382/761] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Oct 12 14:27:30 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 14:27:31.417270 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349660dc-8ddb-49e4-91d8-93a935169e47 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:31.417701 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349660dc-8ddb-49e4-91d8-93a935169e47 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:31.417806 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349660dc-8ddb-49e4-91d8-93a935169e47 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:31.419472 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 380/762] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:31.424720 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ae82c-d056-4d31-b10b-a602130881dc None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:31.425177 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ae82c-d056-4d31-b10b-a602130881dc None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:31.425253 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ae82c-d056-4d31-b10b-a602130881dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:31.444372 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-d45ae82c-d056-4d31-b10b-a602130881dc None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:31.445547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-d45ae82c-d056-4d31-b10b-a602130881dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:31.522361 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 383/763] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:31 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:31.591412 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42a876-7437-45e5-a5d5-ed977d16b65e None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:31.591857 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42a876-7437-45e5-a5d5-ed977d16b65e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:31.591933 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42a876-7437-45e5-a5d5-ed977d16b65e None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:31.593488 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 381/764] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:31.598577 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbd06e-1263-4a93-bfc0-4a402499f384 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:31.598857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbd06e-1263-4a93-bfc0-4a402499f384 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:31.599126 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbd06e-1263-4a93-bfc0-4a402499f384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:31.622476 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-45cbd06e-1263-4a93-bfc0-4a402499f384 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:31.623674 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-45cbd06e-1263-4a93-bfc0-4a402499f384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:31.660946 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:31.694957 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 384/765] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:31 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:31.801621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42a876-7437-45e5-a5d5-ed977d16b65e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:31.815997 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:31.910183 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42a876-7437-45e5-a5d5-ed977d16b65e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:31.945622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197caf26-443b-48c3-82bf-b8ef29f4aabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:31.946572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197caf26-443b-48c3-82bf-b8ef29f4aabd None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:31.947071 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197caf26-443b-48c3-82bf-b8ef29f4aabd None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:31.947571 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197caf26-443b-48c3-82bf-b8ef29f4aabd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:31.948363 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197caf26-443b-48c3-82bf-b8ef29f4aabd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:31.949954 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197caf26-443b-48c3-82bf-b8ef29f4aabd None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-197caf26-443b-48c3-82bf-b8ef29f4aabd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:31.960137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:31.961204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:31.961204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:31.961204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:31.961204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:31.961204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:31.961204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:31.961204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:31.961204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:31.961204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:31.961204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:31.961204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:31.961204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:31.961204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:31.961204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:31.961204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:31.961204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:31.962076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:31.962076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:31.962076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:31.962076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:31.962076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:31.962076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:31.962076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:31.962076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:31.962076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:31.962076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:31.962076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:31.962076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:31.962076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 14:27:31.962076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:31.965072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 382/766] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Oct 12 14:27:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:31.971205 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197caf26-443b-48c3-82bf-b8ef29f4aabd None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:31.971357 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbd06e-1263-4a93-bfc0-4a402499f384 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:32.050041 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:32.077193 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbd06e-1263-4a93-bfc0-4a402499f384 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:32.088030 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197caf26-443b-48c3-82bf-b8ef29f4aabd None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:32.119875 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5883a62-1fc5-4795-a1e1-c395daaf3363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:32.120477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5883a62-1fc5-4795-a1e1-c395daaf3363 None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:32.120754 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5883a62-1fc5-4795-a1e1-c395daaf3363 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:32.121009 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5883a62-1fc5-4795-a1e1-c395daaf3363 None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:32.121661 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5883a62-1fc5-4795-a1e1-c395daaf3363 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:32.123082 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5883a62-1fc5-4795-a1e1-c395daaf3363 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:32.124312 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92df5ba-b5ad-4c43-8b30-05b3bc1fa5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:32.124930 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92df5ba-b5ad-4c43-8b30-05b3bc1fa5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:32.125207 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92df5ba-b5ad-4c43-8b30-05b3bc1fa5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:32.125479 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92df5ba-b5ad-4c43-8b30-05b3bc1fa5b0 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:32.126225 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92df5ba-b5ad-4c43-8b30-05b3bc1fa5b0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:32.126528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92df5ba-b5ad-4c43-8b30-05b3bc1fa5b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:32.127901 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92df5ba-b5ad-4c43-8b30-05b3bc1fa5b0 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:32.133640 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 383/767] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Oct 12 14:27:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:32.139803 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92df5ba-b5ad-4c43-8b30-05b3bc1fa5b0 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:32.144091 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.sql.core [None req-a5883a62-1fc5-4795-a1e1-c395daaf3363 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 12 14:27:32.144091 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 12 14:27:32.144091 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [parameters: {'user_id': '12aad47adf384981927c51762323bd19', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 12 14:27:32.144091 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81566) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 14:27:32.153141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-a5883a62-1fc5-4795-a1e1-c395daaf3363 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 12 14:27:32.153141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:32.153141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 14:27:32.153141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 14:27:32.153141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 14:27:32.153141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 14:27:32.153141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 14:27:32.153141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 14:27:32.153141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 14:27:32.153141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application conn.query(q) Oct 12 14:27:32.153141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 14:27:32.153141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 14:27:32.153141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 14:27:32.153141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application result.read() Oct 12 14:27:32.153141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 14:27:32.154038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application next(self.gen) Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application yield resource Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application next(self.gen) Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise self.value Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application session.commit() Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 14:27:32.155015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self.session.flush() Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application compat.raise_( Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 14:27:32.155885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 14:27:32.156746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 14:27:32.156746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 14:27:32.156746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 12 14:27:32.156746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 12 14:27:32.156746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 14:27:32.156746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 14:27:32.156746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 14:27:32.156746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 14:27:32.156746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 14:27:32.156746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 14:27:32.156746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 14:27:32.156746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 14:27:32.156746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 14:27:32.156746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 14:27:32.156746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 14:27:32.156746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception Oct 12 14:27:32.157592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 14:27:32.157592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 14:27:32.157592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 14:27:32.157592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 14:27:32.157592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 14:27:32.157592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 14:27:32.157592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 14:27:32.157592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application conn.query(q) Oct 12 14:27:32.157592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 14:27:32.157592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 14:27:32.157592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 14:27:32.157592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application result.read() Oct 12 14:27:32.157592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 14:27:32.157592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 14:27:32.157592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 14:27:32.157592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 14:27:32.158381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 14:27:32.158381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 14:27:32.158381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 14:27:32.158381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 14:27:32.158381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 12 14:27:32.158381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 12 14:27:32.158381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [parameters: {'user_id': '12aad47adf384981927c51762323bd19', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 12 14:27:32.158381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 14:27:32.158381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:32.158381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:32.158381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:32.158381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:32.158381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:32.158381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:32.158381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 12 14:27:32.159305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 12 14:27:32.160225 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 12 14:27:32.160225 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 12 14:27:32.160225 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 14:27:32.160225 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 14:27:32.160225 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 12 14:27:32.160225 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:32.168294 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 385/768] 10.222.0.16 () {60 vars in 1131 bytes} [Wed Oct 12 14:27:31 2022] POST /identity/v3/users => generated 160 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:32.169438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5883a62-1fc5-4795-a1e1-c395daaf3363 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:32.211535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:32.246932 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92df5ba-b5ad-4c43-8b30-05b3bc1fa5b0 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:32.282155 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2495923-698f-47f7-804e-df23ba5fde63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:32.282842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2495923-698f-47f7-804e-df23ba5fde63 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:32.283122 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2495923-698f-47f7-804e-df23ba5fde63 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:32.283512 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2495923-698f-47f7-804e-df23ba5fde63 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:32.284048 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2495923-698f-47f7-804e-df23ba5fde63 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:32.284340 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2495923-698f-47f7-804e-df23ba5fde63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:32.285755 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2495923-698f-47f7-804e-df23ba5fde63 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:32.286979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:32.295263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 384/769] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Oct 12 14:27:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:32.297102 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2495923-698f-47f7-804e-df23ba5fde63 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:32.297591 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5883a62-1fc5-4795-a1e1-c395daaf3363 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:32.338216 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3abcb-08a9-4d50-bf47-1d58eda8ce91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:32.339279 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3abcb-08a9-4d50-bf47-1d58eda8ce91 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:32.339693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3abcb-08a9-4d50-bf47-1d58eda8ce91 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:32.340100 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3abcb-08a9-4d50-bf47-1d58eda8ce91 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:32.348145 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3abcb-08a9-4d50-bf47-1d58eda8ce91 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:32.350669 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3abcb-08a9-4d50-bf47-1d58eda8ce91 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:32.363497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-fcc3abcb-08a9-4d50-bf47-1d58eda8ce91 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 14:27:32.363497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:32.363497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:32.363497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:32.363497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:32.363497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:32.363497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:32.363497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:32.363497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:32.363497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:32.363497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:32.363497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:32.363497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:27:32.363497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:27:32.363497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:27:32.363497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:32.364602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:32.366790 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:32.366790 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:32.366790 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:27:32.366790 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:27:32.366790 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:27:32.366790 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:27:32.366790 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:27:32.366790 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:27:32.366790 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 14:27:32.366790 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:32.373817 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 386/770] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:27:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:32.374830 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3abcb-08a9-4d50-bf47-1d58eda8ce91 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:32.404111 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2495923-698f-47f7-804e-df23ba5fde63 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:32.438527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03f4a8-c0a2-4907-9156-3013516398bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:32.439275 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03f4a8-c0a2-4907-9156-3013516398bc None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:32.439626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03f4a8-c0a2-4907-9156-3013516398bc None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:32.439969 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03f4a8-c0a2-4907-9156-3013516398bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:32.440730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c03f4a8-c0a2-4907-9156-3013516398bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:32.442148 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c03f4a8-c0a2-4907-9156-3013516398bc None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:32.467312 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 385/771] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Oct 12 14:27:32 2022] POST /identity/v3/endpoints => generated 325 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 14:27:32.491128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:32.497402 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3abcb-08a9-4d50-bf47-1d58eda8ce91 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:32.549030 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dfc81-3cfe-4895-9a62-65aa8030637d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:32.549636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dfc81-3cfe-4895-9a62-65aa8030637d None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:32.549910 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dfc81-3cfe-4895-9a62-65aa8030637d None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:32.550172 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dfc81-3cfe-4895-9a62-65aa8030637d None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:32.550866 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dfc81-3cfe-4895-9a62-65aa8030637d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:32.551168 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dfc81-3cfe-4895-9a62-65aa8030637d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:32.553171 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dfc81-3cfe-4895-9a62-65aa8030637d None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:32.594418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 387/772] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Oct 12 14:27:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:33.109690 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd4863-7ada-40f9-b4c2-8c7c30e75247 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:33.110211 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd4863-7ada-40f9-b4c2-8c7c30e75247 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:33.110317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd4863-7ada-40f9-b4c2-8c7c30e75247 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:33.111737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 386/773] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:33.117738 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f25cb-7acb-4d65-8782-0d6af7ff663e None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:33.118021 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f25cb-7acb-4d65-8782-0d6af7ff663e None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:33.118294 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f25cb-7acb-4d65-8782-0d6af7ff663e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:33.142820 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-517f25cb-7acb-4d65-8782-0d6af7ff663e None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:33.144416 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-517f25cb-7acb-4d65-8782-0d6af7ff663e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:33.219882 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 388/774] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:33 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 14:27:33.274228 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823812d9-3e4a-488c-8b22-9521457555ec None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:33.274552 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823812d9-3e4a-488c-8b22-9521457555ec None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:33.274775 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823812d9-3e4a-488c-8b22-9521457555ec None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:33.276517 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 387/775] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:33.281520 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c23e3-7a8f-454e-85f5-3c97e62918c9 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:33.281807 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c23e3-7a8f-454e-85f5-3c97e62918c9 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:33.282076 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c23e3-7a8f-454e-85f5-3c97e62918c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:33.300386 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-3b5c23e3-7a8f-454e-85f5-3c97e62918c9 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:33.301554 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-3b5c23e3-7a8f-454e-85f5-3c97e62918c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:33.369594 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 389/776] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:33 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:33.497121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823812d9-3e4a-488c-8b22-9521457555ec None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:33.557959 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:33.602815 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823812d9-3e4a-488c-8b22-9521457555ec None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:33.635363 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c23e3-7a8f-454e-85f5-3c97e62918c9 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:33.638393 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a14401-eb34-44d2-a95c-c9e9fc05ad2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:33.639031 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a14401-eb34-44d2-a95c-c9e9fc05ad2d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:33.639351 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a14401-eb34-44d2-a95c-c9e9fc05ad2d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:33.639620 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a14401-eb34-44d2-a95c-c9e9fc05ad2d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:33.640205 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a14401-eb34-44d2-a95c-c9e9fc05ad2d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:33.641636 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a14401-eb34-44d2-a95c-c9e9fc05ad2d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:33.645424 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-56a14401-eb34-44d2-a95c-c9e9fc05ad2d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:33.651436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:33.652432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:33.652432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:33.652432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:33.652432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:33.652432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:33.652432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:33.652432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:33.652432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:33.652432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:33.652432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:33.652432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:33.652432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:33.652432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:33.652432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:33.652432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:33.652432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:33.656178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:33.656178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:33.656178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:33.656178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:33.656178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:33.656178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:33.656178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:33.656178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:33.656178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:33.656178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:33.656178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:33.656178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:33.656178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 14:27:33.656178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:33.656178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 388/777] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:27:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:33.658548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a14401-eb34-44d2-a95c-c9e9fc05ad2d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:33.744304 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c23e3-7a8f-454e-85f5-3c97e62918c9 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:33.765216 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a14401-eb34-44d2-a95c-c9e9fc05ad2d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:33.782658 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc78b86-e811-4df0-9b81-093b80307901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:33.783599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc78b86-e811-4df0-9b81-093b80307901 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:33.783877 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc78b86-e811-4df0-9b81-093b80307901 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:33.784144 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc78b86-e811-4df0-9b81-093b80307901 None admin] PATH_INFO: `/v3/services/image` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:33.784726 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc78b86-e811-4df0-9b81-093b80307901 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:33.786535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc78b86-e811-4df0-9b81-093b80307901 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:33.788741 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-1fc78b86-e811-4df0-9b81-093b80307901 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:33.795558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:33.796543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:33.796543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:33.796543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:33.796543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:33.796543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:33.796543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:33.796543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:27:33.796543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:27:33.796543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:27:33.796543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:27:33.796543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:33.796543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:33.796543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:33.796543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:33.796543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:33.796543 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:33.798038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:33.798038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:33.798038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:33.798038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:33.798038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:33.798038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:33.798038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:33.798038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:33.798038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:33.798038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:33.798038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:27:33.798038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:27:33.798038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 14:27:33.798038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:33.798038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 390/778] 10.222.0.16 () {56 vars in 1105 bytes} [Wed Oct 12 14:27:33 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:33.802600 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc78b86-e811-4df0-9b81-093b80307901 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:33.804775 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75a864-feab-47e2-b6e1-5951d4e0e6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:33.805386 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75a864-feab-47e2-b6e1-5951d4e0e6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:33.805659 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75a864-feab-47e2-b6e1-5951d4e0e6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:33.805923 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75a864-feab-47e2-b6e1-5951d4e0e6d9 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:33.806550 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75a864-feab-47e2-b6e1-5951d4e0e6d9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:33.806848 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75a864-feab-47e2-b6e1-5951d4e0e6d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:33.808239 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75a864-feab-47e2-b6e1-5951d4e0e6d9 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:33.813456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 389/779] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Oct 12 14:27:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:33.819265 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75a864-feab-47e2-b6e1-5951d4e0e6d9 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:33.874406 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:33.905113 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc78b86-e811-4df0-9b81-093b80307901 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:33.928443 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75a864-feab-47e2-b6e1-5951d4e0e6d9 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:33.939749 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245fe3d-8a28-47f2-8789-fc5ae0f85391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:33.940297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245fe3d-8a28-47f2-8789-fc5ae0f85391 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:33.940569 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245fe3d-8a28-47f2-8789-fc5ae0f85391 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:33.940834 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245fe3d-8a28-47f2-8789-fc5ae0f85391 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:33.941446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245fe3d-8a28-47f2-8789-fc5ae0f85391 None admin] RBAC: Adding query filter params (name=image) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:33.941755 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245fe3d-8a28-47f2-8789-fc5ae0f85391 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:33.943171 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245fe3d-8a28-47f2-8789-fc5ae0f85391 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:33.948651 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 391/780] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:33.953881 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245fe3d-8a28-47f2-8789-fc5ae0f85391 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:33.969316 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397a8ea-8923-4575-acae-dd23580442e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:33.970037 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397a8ea-8923-4575-acae-dd23580442e6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:33.970309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397a8ea-8923-4575-acae-dd23580442e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:33.970647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397a8ea-8923-4575-acae-dd23580442e6 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:33.971216 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397a8ea-8923-4575-acae-dd23580442e6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:33.971517 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397a8ea-8923-4575-acae-dd23580442e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:33.973055 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397a8ea-8923-4575-acae-dd23580442e6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:33.982650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 390/781] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Oct 12 14:27:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:33.983617 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397a8ea-8923-4575-acae-dd23580442e6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:34.037986 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:34.061582 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245fe3d-8a28-47f2-8789-fc5ae0f85391 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:34.091521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397a8ea-8923-4575-acae-dd23580442e6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:34.096731 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0641b-8638-4ee7-a951-ad2f6eb0b132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:34.097338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0641b-8638-4ee7-a951-ad2f6eb0b132 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:34.097612 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0641b-8638-4ee7-a951-ad2f6eb0b132 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:34.097877 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0641b-8638-4ee7-a951-ad2f6eb0b132 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:34.098645 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0641b-8638-4ee7-a951-ad2f6eb0b132 None admin] RBAC: Adding query filter params (type=image) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:34.098770 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0641b-8638-4ee7-a951-ad2f6eb0b132 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:34.100911 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0641b-8638-4ee7-a951-ad2f6eb0b132 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:34.103416 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:34.111341 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 392/782] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:34.113176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0641b-8638-4ee7-a951-ad2f6eb0b132 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:34.132501 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599a7ee-2044-428b-9346-5adacb8d76b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:34.133353 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599a7ee-2044-428b-9346-5adacb8d76b7 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:34.134013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599a7ee-2044-428b-9346-5adacb8d76b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:34.134287 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599a7ee-2044-428b-9346-5adacb8d76b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:34.134946 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599a7ee-2044-428b-9346-5adacb8d76b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:34.136867 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599a7ee-2044-428b-9346-5adacb8d76b7 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:34.159041 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 391/783] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Oct 12 14:27:33 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 14:27:34.231519 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0641b-8638-4ee7-a951-ad2f6eb0b132 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:34.277032 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e021e0-cb97-40d0-ae1c-2d120da59fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:34.277650 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e021e0-cb97-40d0-ae1c-2d120da59fbf None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:34.277927 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e021e0-cb97-40d0-ae1c-2d120da59fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:34.278193 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e021e0-cb97-40d0-ae1c-2d120da59fbf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:34.278864 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e021e0-cb97-40d0-ae1c-2d120da59fbf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:34.280329 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e021e0-cb97-40d0-ae1c-2d120da59fbf None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:34.311183 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 393/784] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Oct 12 14:27:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 14:27:34.710547 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303e748-0441-47c2-8e3a-a0dcf3be6309 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:34.710911 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303e748-0441-47c2-8e3a-a0dcf3be6309 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:34.711086 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303e748-0441-47c2-8e3a-a0dcf3be6309 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:34.714162 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 392/785] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:34.720913 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3810-06f6-422d-9784-ac9b9e9438b1 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:34.720913 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3810-06f6-422d-9784-ac9b9e9438b1 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:34.720913 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3810-06f6-422d-9784-ac9b9e9438b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:34.738498 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-9ddb3810-06f6-422d-9784-ac9b9e9438b1 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:34.739699 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-9ddb3810-06f6-422d-9784-ac9b9e9438b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:34.788696 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:34.819200 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 394/786] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:35.093152 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303e748-0441-47c2-8e3a-a0dcf3be6309 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:35.200053 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303e748-0441-47c2-8e3a-a0dcf3be6309 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:35.209586 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:35.240719 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264aa2ec-5776-414a-8532-b71bf91f781c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:35.241557 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264aa2ec-5776-414a-8532-b71bf91f781c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:35.241828 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264aa2ec-5776-414a-8532-b71bf91f781c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:35.242088 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264aa2ec-5776-414a-8532-b71bf91f781c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:35.243208 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264aa2ec-5776-414a-8532-b71bf91f781c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:35.244800 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264aa2ec-5776-414a-8532-b71bf91f781c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:35.247176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 393/787] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:35.252271 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb3810-06f6-422d-9784-ac9b9e9438b1 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:35.350809 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb3810-06f6-422d-9784-ac9b9e9438b1 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:35.383416 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e26b47-86ff-4561-a1eb-0e1a7ff904d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:35.384013 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e26b47-86ff-4561-a1eb-0e1a7ff904d8 None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:35.384284 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e26b47-86ff-4561-a1eb-0e1a7ff904d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:35.384552 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e26b47-86ff-4561-a1eb-0e1a7ff904d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:35.385191 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e26b47-86ff-4561-a1eb-0e1a7ff904d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:35.386624 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e26b47-86ff-4561-a1eb-0e1a7ff904d8 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:35.419178 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 395/788] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Oct 12 14:27:35 2022] POST /identity/v3/projects => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:27:36.424022 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e636f-505d-4fcc-9839-6996c6d14f97 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:36.424546 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e636f-505d-4fcc-9839-6996c6d14f97 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:36.424546 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e636f-505d-4fcc-9839-6996c6d14f97 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:36.426129 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 394/789] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:36.431255 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63e182-d571-475b-8957-2422f28dd85b None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:36.431705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63e182-d571-475b-8957-2422f28dd85b None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:36.431794 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63e182-d571-475b-8957-2422f28dd85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:36.451127 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-3c63e182-d571-475b-8957-2422f28dd85b None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:36.452358 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-3c63e182-d571-475b-8957-2422f28dd85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:36.477274 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:36.503957 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb911c26-3cc5-4cd9-8b22-f329556990ba None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:36.504274 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb911c26-3cc5-4cd9-8b22-f329556990ba None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:36.504498 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb911c26-3cc5-4cd9-8b22-f329556990ba None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:36.505805 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 395/790] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:27:36.511679 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d18ec-71fc-4208-bcfc-76084d4dd6c1 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:36.511951 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d18ec-71fc-4208-bcfc-76084d4dd6c1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:36.512215 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d18ec-71fc-4208-bcfc-76084d4dd6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:36.527241 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 396/791] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:36.530994 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-411d18ec-71fc-4208-bcfc-76084d4dd6c1 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:36.532182 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-411d18ec-71fc-4208-bcfc-76084d4dd6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:36.606189 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 396/792] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 14:27:36.796099 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63e182-d571-475b-8957-2422f28dd85b None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:36.877641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d18ec-71fc-4208-bcfc-76084d4dd6c1 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:36.898017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63e182-d571-475b-8957-2422f28dd85b None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:36.933492 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4813e-2867-40ef-8919-07ac84e23455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:36.934525 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4813e-2867-40ef-8919-07ac84e23455 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:36.934804 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4813e-2867-40ef-8919-07ac84e23455 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:36.935069 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4813e-2867-40ef-8919-07ac84e23455 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:36.945711 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4813e-2867-40ef-8919-07ac84e23455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:27:36.950545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-60d4813e-2867-40ef-8919-07ac84e23455 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 14:27:36.950545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:36.950545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:36.950545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:36.950545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:36.950545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:36.950545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:36.950545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:36.950545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:36.950545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:36.950545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:36.950545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:36.950545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:27:36.950545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:27:36.950545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:27:36.950545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:27:36.951533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:27:36.951533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:27:36.951533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:27:36.951533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:27:36.951533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:36.951533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:36.951533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:36.951533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:36.951533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:36.951533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:36.951533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:36.951533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:36.951533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:36.951533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:36.951533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:36.951533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:36.952536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:36.952536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:36.952536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:36.952536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:36.952536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:27:36.952536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:27:36.952536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:27:36.952536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:27:36.952536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:27:36.952536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:27:36.952536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 14:27:36.952536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:36.956642 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 397/793] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Oct 12 14:27:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:36.957497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4813e-2867-40ef-8919-07ac84e23455 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:36.979466 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:36.988761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d18ec-71fc-4208-bcfc-76084d4dd6c1 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:36.992856 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:37.029072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ae52f-8b55-4291-a138-29bd0dd4db39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:37.029675 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae52f-8b55-4291-a138-29bd0dd4db39 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:37.029947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae52f-8b55-4291-a138-29bd0dd4db39 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:37.030210 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae52f-8b55-4291-a138-29bd0dd4db39 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:37.030864 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ae52f-8b55-4291-a138-29bd0dd4db39 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:37.032278 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ae52f-8b55-4291-a138-29bd0dd4db39 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:37.041964 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.sql.core [None req-4d7ae52f-8b55-4291-a138-29bd0dd4db39 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 14:27:37.041964 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 14:27:37.041964 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [parameters: {'id': '075aa4c453904cd580454c95a81af4c3', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 12 14:27:37.041964 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81565) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 14:27:37.049967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-4d7ae52f-8b55-4291-a138-29bd0dd4db39 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 12 14:27:37.049967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:37.049967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 14:27:37.049967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 14:27:37.049967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 14:27:37.049967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 14:27:37.049967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 14:27:37.049967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 14:27:37.049967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 14:27:37.049967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application conn.query(q) Oct 12 14:27:37.049967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 14:27:37.049967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 14:27:37.049967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 14:27:37.049967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result.read() Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 12 14:27:37.050948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application next(self.gen) Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application yield resource Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application next(self.gen) Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise self.value Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application session.commit() Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 14:27:37.052607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.session.flush() Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application compat.raise_( Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 14:27:37.054382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 14:27:37.056268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 14:27:37.056268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 14:27:37.056268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 14:27:37.056268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 12 14:27:37.056268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 12 14:27:37.056268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 14:27:37.056268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 14:27:37.056268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 14:27:37.056268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 14:27:37.056268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 14:27:37.056268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 14:27:37.056268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 14:27:37.056268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 14:27:37.056268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 14:27:37.056268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application conn.query(q) Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result.read() Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 14:27:37.058121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 14:27:37.060689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 14:27:37.060689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 14:27:37.060689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 14:27:37.060689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 14:27:37.060689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 14:27:37.060689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 14:27:37.060689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 14:27:37.060689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [parameters: {'id': '075aa4c453904cd580454c95a81af4c3', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 12 14:27:37.060689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 14:27:37.060689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:37.060689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:37.060689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:37.060689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:37.060689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 12 14:27:37.060689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:37.063333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:37.066293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 12 14:27:37.066293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 12 14:27:37.066293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:37.066293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:37.066293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 12 14:27:37.066293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 12 14:27:37.066293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:37.066293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:37.066293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 12 14:27:37.066293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 12 14:27:37.066293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 12 14:27:37.066293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:37.066293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 397/794] 10.222.0.16 () {60 vars in 1137 bytes} [Wed Oct 12 14:27:36 2022] POST /identity/v3/domains => generated 190 bytes in 174 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:37.066293 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ae52f-8b55-4291-a138-29bd0dd4db39 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:37.068447 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4813e-2867-40ef-8919-07ac84e23455 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:37.104941 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221f3f1-718b-46fb-8256-e4ffbe59b125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:37.105983 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221f3f1-718b-46fb-8256-e4ffbe59b125 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:37.106161 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221f3f1-718b-46fb-8256-e4ffbe59b125 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:37.106429 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221f3f1-718b-46fb-8256-e4ffbe59b125 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:37.107095 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221f3f1-718b-46fb-8256-e4ffbe59b125 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:37.107423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221f3f1-718b-46fb-8256-e4ffbe59b125 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:37.108799 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221f3f1-718b-46fb-8256-e4ffbe59b125 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:37.121702 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 398/795] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Oct 12 14:27:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:37.122580 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221f3f1-718b-46fb-8256-e4ffbe59b125 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:37.169161 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ae52f-8b55-4291-a138-29bd0dd4db39 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:37.204625 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b107ca-f8cb-4598-be42-e9fa788a9364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:37.205349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b107ca-f8cb-4598-be42-e9fa788a9364 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:37.205584 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b107ca-f8cb-4598-be42-e9fa788a9364 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:37.205847 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b107ca-f8cb-4598-be42-e9fa788a9364 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:37.211606 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:37.218430 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b107ca-f8cb-4598-be42-e9fa788a9364 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:37.219901 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b107ca-f8cb-4598-be42-e9fa788a9364 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:37.230742 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221f3f1-718b-46fb-8256-e4ffbe59b125 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:37.236666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-17b107ca-f8cb-4598-be42-e9fa788a9364 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:37.236666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:37.236666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:37.236666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:37.236666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:37.236666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:37.236666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:37.236666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:37.236666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:37.236666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:37.236666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:37.236666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:37.236666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:37.236666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:37.236666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:37.236666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:37.237576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:37.238666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:37.238666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:37.238666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:37.238666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 398/796] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:37.244318 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b107ca-f8cb-4598-be42-e9fa788a9364 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:37.266589 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:37.268605 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87bce1-6444-4e0d-8920-a492d97c290f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:37.269470 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87bce1-6444-4e0d-8920-a492d97c290f None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:37.269735 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87bce1-6444-4e0d-8920-a492d97c290f None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:37.270020 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87bce1-6444-4e0d-8920-a492d97c290f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:37.282262 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87bce1-6444-4e0d-8920-a492d97c290f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:37.283744 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87bce1-6444-4e0d-8920-a492d97c290f None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:37.298508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-ff87bce1-6444-4e0d-8920-a492d97c290f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:37.298508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:37.298508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:37.298508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:37.298508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:37.298508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:37.298508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:37.298508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:37.298508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:37.298508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:37.298508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:37.298508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:37.298508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:37.298508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:37.298508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:37.298508 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:37.299570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:37.300675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:37.300675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:37.300675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:37.305218 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 399/797] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:37.306044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87bce1-6444-4e0d-8920-a492d97c290f None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:37.353012 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b107ca-f8cb-4598-be42-e9fa788a9364 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:37.391829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfba1ae-c95b-4b2f-af7e-2989d14ec3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:37.392949 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfba1ae-c95b-4b2f-af7e-2989d14ec3c4 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:37.393399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfba1ae-c95b-4b2f-af7e-2989d14ec3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:37.393815 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfba1ae-c95b-4b2f-af7e-2989d14ec3c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:37.394607 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfba1ae-c95b-4b2f-af7e-2989d14ec3c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:37.395497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfba1ae-c95b-4b2f-af7e-2989d14ec3c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:37.397567 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfba1ae-c95b-4b2f-af7e-2989d14ec3c4 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:37.409614 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87bce1-6444-4e0d-8920-a492d97c290f None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:37.419242 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 399/798] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:37.448408 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0986780-d717-482c-9f92-83b8b63b1efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:37.449071 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0986780-d717-482c-9f92-83b8b63b1efb None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:37.449302 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0986780-d717-482c-9f92-83b8b63b1efb None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:37.449574 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0986780-d717-482c-9f92-83b8b63b1efb None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:37.450245 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0986780-d717-482c-9f92-83b8b63b1efb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:37.450539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0986780-d717-482c-9f92-83b8b63b1efb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:37.452039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0986780-d717-482c-9f92-83b8b63b1efb None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:37.454096 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:37.472814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 400/799] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:37.474459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfba1ae-c95b-4b2f-af7e-2989d14ec3c4 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:37.569861 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:37.597473 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfba1ae-c95b-4b2f-af7e-2989d14ec3c4 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:37.612641 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ffd5c-896b-4bf2-a8e0-3a856d13179a None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:37.612966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ffd5c-896b-4bf2-a8e0-3a856d13179a None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:37.613183 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ffd5c-896b-4bf2-a8e0-3a856d13179a None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:37.614644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 401/800] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:27:37.620459 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61cae7-e494-4cc6-871e-cd067a422bcb None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:37.620738 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61cae7-e494-4cc6-871e-cd067a422bcb None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:37.621003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61cae7-e494-4cc6-871e-cd067a422bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:37.639614 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e5b7f-1714-44ec-ade9-e05b898d2225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:37.640259 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e5b7f-1714-44ec-ade9-e05b898d2225 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:37.640528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e5b7f-1714-44ec-ade9-e05b898d2225 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:37.640790 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e5b7f-1714-44ec-ade9-e05b898d2225 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:37.650109 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e5b7f-1714-44ec-ade9-e05b898d2225 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:37.651906 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e5b7f-1714-44ec-ade9-e05b898d2225 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:37.667034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-a90e5b7f-1714-44ec-ade9-e05b898d2225 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 14:27:37.667034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:37.667034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:37.667034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:37.667034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:37.667034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:37.667034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:37.667034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:37.667034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:37.667034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:37.667034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:37.667034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:37.667034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:27:37.667034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:27:37.667034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:27:37.667034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:37.668077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:37.669103 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:37.669103 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:37.669103 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:27:37.669103 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:27:37.669103 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:27:37.669103 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:27:37.669103 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:27:37.669103 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:27:37.669103 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 14:27:37.669103 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:37.669103 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 400/801] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Oct 12 14:27:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:37.674458 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e5b7f-1714-44ec-ade9-e05b898d2225 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:37.764291 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-fc61cae7-e494-4cc6-871e-cd067a422bcb None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:37.765489 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-fc61cae7-e494-4cc6-871e-cd067a422bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:37.790094 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e5b7f-1714-44ec-ade9-e05b898d2225 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:37.824240 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6d251-b849-4538-805e-9d34a48c6996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:37.824762 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6d251-b849-4538-805e-9d34a48c6996 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:37.825034 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6d251-b849-4538-805e-9d34a48c6996 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:37.825296 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6d251-b849-4538-805e-9d34a48c6996 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:37.826007 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6d251-b849-4538-805e-9d34a48c6996 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:37.826304 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6d251-b849-4538-805e-9d34a48c6996 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:37.827748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6d251-b849-4538-805e-9d34a48c6996 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:37.831330 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 402/802] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 14:27:37.836662 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:37.844656 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 401/803] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Oct 12 14:27:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:37.850980 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61cae7-e494-4cc6-871e-cd067a422bcb None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:37.946208 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61cae7-e494-4cc6-871e-cd067a422bcb None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:37.968677 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:37.979942 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35dfdc2-4ea3-4b51-83f6-d75751ecb6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:37.980558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dfdc2-4ea3-4b51-83f6-d75751ecb6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:37.980832 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dfdc2-4ea3-4b51-83f6-d75751ecb6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:37.981098 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dfdc2-4ea3-4b51-83f6-d75751ecb6e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:37.991313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35dfdc2-4ea3-4b51-83f6-d75751ecb6e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:37.992855 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35dfdc2-4ea3-4b51-83f6-d75751ecb6e8 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:38.006984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-b35dfdc2-4ea3-4b51-83f6-d75751ecb6e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:38.006984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:38.006984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:38.006984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:38.006984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:38.006984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:38.006984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:38.006984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:38.006984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:38.006984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:38.006984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:38.006984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:38.006984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:38.006984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:38.006984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:38.006984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:38.013001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:38.013958 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:38.013958 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:38.013958 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:38.013958 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 403/804] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:38.014211 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6d251-b849-4538-805e-9d34a48c6996 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:38.099135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35dfdc2-4ea3-4b51-83f6-d75751ecb6e8 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:38.109672 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6d251-b849-4538-805e-9d34a48c6996 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:38.142323 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20a2d7-e523-4fd4-b58a-f43307abdecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:38.142922 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20a2d7-e523-4fd4-b58a-f43307abdecd None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:38.143221 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20a2d7-e523-4fd4-b58a-f43307abdecd None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:38.143490 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20a2d7-e523-4fd4-b58a-f43307abdecd None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:38.144122 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20a2d7-e523-4fd4-b58a-f43307abdecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:38.144421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20a2d7-e523-4fd4-b58a-f43307abdecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:38.145760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20a2d7-e523-4fd4-b58a-f43307abdecd None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:38.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 402/805] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:38.162318 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20a2d7-e523-4fd4-b58a-f43307abdecd None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:38.197628 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35dfdc2-4ea3-4b51-83f6-d75751ecb6e8 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:38.219328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:38.232566 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e90d7-e390-4a62-a279-d2ea72403a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:38.233191 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e90d7-e390-4a62-a279-d2ea72403a93 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:38.233464 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e90d7-e390-4a62-a279-d2ea72403a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:38.233727 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e90d7-e390-4a62-a279-d2ea72403a93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:38.234817 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e90d7-e390-4a62-a279-d2ea72403a93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:38.236307 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e90d7-e390-4a62-a279-d2ea72403a93 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:38.237875 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 404/806] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:38.244021 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e90d7-e390-4a62-a279-d2ea72403a93 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:38.260435 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20a2d7-e523-4fd4-b58a-f43307abdecd None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:38.289487 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:38.293092 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106f30a-91fd-45bb-9faf-6b437bb2e13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:38.293712 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106f30a-91fd-45bb-9faf-6b437bb2e13b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:38.293976 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106f30a-91fd-45bb-9faf-6b437bb2e13b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:38.294239 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106f30a-91fd-45bb-9faf-6b437bb2e13b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:38.300624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106f30a-91fd-45bb-9faf-6b437bb2e13b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:38.302031 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106f30a-91fd-45bb-9faf-6b437bb2e13b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:38.313626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-a106f30a-91fd-45bb-9faf-6b437bb2e13b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:38.313626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:38.313626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:38.313626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:38.313626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:38.313626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:38.313626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:38.313626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:38.313626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:38.313626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:38.313626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:38.313626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:38.313626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:27:38.313626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:27:38.313626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:38.314634 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:38.315587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:27:38.315587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:27:38.315587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:27:38.315587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:27:38.315587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:27:38.315587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:27:38.315587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:38.315587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:38.315587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 403/807] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:27:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:38.321407 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106f30a-91fd-45bb-9faf-6b437bb2e13b None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:38.348072 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e90d7-e390-4a62-a279-d2ea72403a93 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:38.382200 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfef19-70f5-4080-b7ac-0dcc96d477db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:38.382736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfef19-70f5-4080-b7ac-0dcc96d477db None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:38.383038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfef19-70f5-4080-b7ac-0dcc96d477db None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:38.383348 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfef19-70f5-4080-b7ac-0dcc96d477db None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:38.383995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdfef19-70f5-4080-b7ac-0dcc96d477db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:38.385420 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdfef19-70f5-4080-b7ac-0dcc96d477db None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:38.406301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 405/808] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:27:38 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 14:27:38.420137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106f30a-91fd-45bb-9faf-6b437bb2e13b None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:38.439526 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:38.459050 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf544615-870a-4dfb-ba10-ce9ecc7d6f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:38.459864 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf544615-870a-4dfb-ba10-ce9ecc7d6f85 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:38.460448 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf544615-870a-4dfb-ba10-ce9ecc7d6f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:38.460655 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf544615-870a-4dfb-ba10-ce9ecc7d6f85 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:38.461373 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf544615-870a-4dfb-ba10-ce9ecc7d6f85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:38.461675 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf544615-870a-4dfb-ba10-ce9ecc7d6f85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:38.463174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf544615-870a-4dfb-ba10-ce9ecc7d6f85 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:38.476836 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 404/809] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:27:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:38.483169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfef19-70f5-4080-b7ac-0dcc96d477db None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:38.591418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:38.594656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfef19-70f5-4080-b7ac-0dcc96d477db None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:38.630984 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143f263-94a2-4c36-a619-f81f88f7242f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:38.632333 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143f263-94a2-4c36-a619-f81f88f7242f None admin] REQUEST_METHOD: `PUT` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:38.632725 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143f263-94a2-4c36-a619-f81f88f7242f None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:38.632994 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143f263-94a2-4c36-a619-f81f88f7242f None admin] PATH_INFO: `/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/d0aeed5f32a1477dbd9d3e934669705b/roles/f074c279fe6544eba0c9d54f5150fbd6` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:38.635208 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143f263-94a2-4c36-a619-f81f88f7242f None admin] RBAC: Authorizing `identity:create_grant(project_id=720b11237d2e48128c171be45fcd0ad0, user_id=d0aeed5f32a1477dbd9d3e934669705b, role_id=f074c279fe6544eba0c9d54f5150fbd6)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:38.636650 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143f263-94a2-4c36-a619-f81f88f7242f None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:38.650265 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 406/810] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:27:38 2022] PUT /identity/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/d0aeed5f32a1477dbd9d3e934669705b/roles/f074c279fe6544eba0c9d54f5150fbd6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:27:40.572899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc2f09-3f4d-478e-bba5-70ab74e76c21 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:40.573430 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc2f09-3f4d-478e-bba5-70ab74e76c21 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:40.573662 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc2f09-3f4d-478e-bba5-70ab74e76c21 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:40.575239 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 405/811] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:40.580380 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbf4e7-b087-4cfb-84e2-efd37a3b6945 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:40.580768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbf4e7-b087-4cfb-84e2-efd37a3b6945 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:40.580962 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbf4e7-b087-4cfb-84e2-efd37a3b6945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:40.598678 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-2bdbf4e7-b087-4cfb-84e2-efd37a3b6945 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:40.599875 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-2bdbf4e7-b087-4cfb-84e2-efd37a3b6945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:40.665242 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 407/812] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:40.796743 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174d97a-cc19-4a18-a15f-0b4740fcc7c3 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:40.797084 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174d97a-cc19-4a18-a15f-0b4740fcc7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:40.797414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174d97a-cc19-4a18-a15f-0b4740fcc7c3 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:40.798745 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 406/813] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:40.804935 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79aece7-dd4c-4ef1-b3e0-8ea11b09d209 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:40.805217 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79aece7-dd4c-4ef1-b3e0-8ea11b09d209 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:40.805481 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79aece7-dd4c-4ef1-b3e0-8ea11b09d209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:40.824323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-d79aece7-dd4c-4ef1-b3e0-8ea11b09d209 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:40.825400 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-d79aece7-dd4c-4ef1-b3e0-8ea11b09d209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:40.889156 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 408/814] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:40.933600 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174d97a-cc19-4a18-a15f-0b4740fcc7c3 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:41.031728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174d97a-cc19-4a18-a15f-0b4740fcc7c3 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:41.063663 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee05426-a1d0-45ef-810d-1317bcfab756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:41.064276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee05426-a1d0-45ef-810d-1317bcfab756 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:41.064550 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee05426-a1d0-45ef-810d-1317bcfab756 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:41.064815 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee05426-a1d0-45ef-810d-1317bcfab756 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:41.075139 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee05426-a1d0-45ef-810d-1317bcfab756 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:27:41.080003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-eee05426-a1d0-45ef-810d-1317bcfab756 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:27:41.080003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:41.080003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:41.080003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:41.080003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:41.080003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:41.080003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:41.080003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:41.080003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:41.080003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:41.080003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:41.080003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:41.080003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:27:41.080003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:27:41.080003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:27:41.080003 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:27:41.083184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:27:41.083184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:27:41.083184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:27:41.083184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:27:41.083184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:41.083184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:41.083184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:41.083184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:41.083184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:41.083184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:41.083184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:41.083184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:41.083184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:41.083184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:41.083184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:41.083184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:41.084279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:41.084279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:41.084279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:41.084279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:41.084279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:27:41.084279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:27:41.084279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:27:41.084279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:27:41.084279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:27:41.084279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:27:41.084279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:27:41.084279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:41.084279 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 407/815] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:27:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:41.087737 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79aece7-dd4c-4ef1-b3e0-8ea11b09d209 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:41.108341 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:41.160955 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee05426-a1d0-45ef-810d-1317bcfab756 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:41.188460 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79aece7-dd4c-4ef1-b3e0-8ea11b09d209 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:41.203963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:41.224569 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aef31e-8d0b-4fe0-80f5-da67c9ab8417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:41.225708 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aef31e-8d0b-4fe0-80f5-da67c9ab8417 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:41.226067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aef31e-8d0b-4fe0-80f5-da67c9ab8417 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:41.226353 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aef31e-8d0b-4fe0-80f5-da67c9ab8417 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:41.227099 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aef31e-8d0b-4fe0-80f5-da67c9ab8417 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:41.227434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aef31e-8d0b-4fe0-80f5-da67c9ab8417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:41.228803 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aef31e-8d0b-4fe0-80f5-da67c9ab8417 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:41.236773 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 409/816] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:27:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:41.243275 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aef31e-8d0b-4fe0-80f5-da67c9ab8417 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:41.264412 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee05426-a1d0-45ef-810d-1317bcfab756 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:41.299721 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3171a-67f4-44ee-b661-3fc49cd7e7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:41.300339 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3171a-67f4-44ee-b661-3fc49cd7e7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:41.300609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3171a-67f4-44ee-b661-3fc49cd7e7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:41.300875 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3171a-67f4-44ee-b661-3fc49cd7e7c6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:41.311116 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3171a-67f4-44ee-b661-3fc49cd7e7c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:27:41.315928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-daa3171a-67f4-44ee-b661-3fc49cd7e7c6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 14:27:41.315928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:41.315928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:41.315928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:41.315928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:41.315928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:41.315928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:41.315928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:41.315928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:41.315928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:41.315928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:41.315928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:41.315928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:27:41.315928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:27:41.315928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:27:41.315928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:27:41.316904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:27:41.316904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:27:41.316904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:27:41.316904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:27:41.316904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:41.316904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:41.316904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:41.316904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:41.316904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:41.316904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:41.316904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:41.316904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:41.316904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:41.316904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:41.316904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:41.316904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:41.317734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:41.317734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:41.317734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:41.317734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:41.317734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:27:41.317734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:27:41.317734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:27:41.317734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:27:41.317734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:27:41.317734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:27:41.317734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 14:27:41.317734 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:41.319286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 408/817] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Oct 12 14:27:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:41.323496 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3171a-67f4-44ee-b661-3fc49cd7e7c6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:41.323760 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:41.343601 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aef31e-8d0b-4fe0-80f5-da67c9ab8417 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:41.376397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb7f9f-a9d3-44a5-a258-b1ea952ee4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:41.377026 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb7f9f-a9d3-44a5-a258-b1ea952ee4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:41.377299 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb7f9f-a9d3-44a5-a258-b1ea952ee4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:41.377570 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb7f9f-a9d3-44a5-a258-b1ea952ee4f2 None admin] PATH_INFO: `/v3/users/fb3d5e300ce94a68a7cee589212a6ddf` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:41.390427 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb7f9f-a9d3-44a5-a258-b1ea952ee4f2 None admin] RBAC: Authorizing `identity:get_user(user_id=fb3d5e300ce94a68a7cee589212a6ddf)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:41.391880 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb7f9f-a9d3-44a5-a258-b1ea952ee4f2 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:41.393497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 410/818] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:27:41 2022] GET /identity/v3/users/fb3d5e300ce94a68a7cee589212a6ddf => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:41.399875 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb7f9f-a9d3-44a5-a258-b1ea952ee4f2 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:41.419934 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:41.424050 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3171a-67f4-44ee-b661-3fc49cd7e7c6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:41.457841 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd60fe-abfa-4b8b-8382-60b236b21117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:41.458441 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd60fe-abfa-4b8b-8382-60b236b21117 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:41.458717 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd60fe-abfa-4b8b-8382-60b236b21117 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:41.459239 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd60fe-abfa-4b8b-8382-60b236b21117 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:41.460353 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd60fe-abfa-4b8b-8382-60b236b21117 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:41.460671 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd60fe-abfa-4b8b-8382-60b236b21117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:41.462010 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd60fe-abfa-4b8b-8382-60b236b21117 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:41.469849 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 409/819] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Oct 12 14:27:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:41.475968 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd60fe-abfa-4b8b-8382-60b236b21117 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:41.495793 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb7f9f-a9d3-44a5-a258-b1ea952ee4f2 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:41.529724 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f8fa7-abec-4040-949a-eef6fb67dca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:41.530346 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f8fa7-abec-4040-949a-eef6fb67dca5 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:41.530622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f8fa7-abec-4040-949a-eef6fb67dca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:41.530890 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f8fa7-abec-4040-949a-eef6fb67dca5 None admin] PATH_INFO: `/v3/projects/743463bbeb374c089f42bb822d0fb893` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:41.532037 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f8fa7-abec-4040-949a-eef6fb67dca5 None admin] RBAC: Authorizing `identity:get_project(project_id=743463bbeb374c089f42bb822d0fb893)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:41.533455 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f8fa7-abec-4040-949a-eef6fb67dca5 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:41.535229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 411/820] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:27:41 2022] GET /identity/v3/projects/743463bbeb374c089f42bb822d0fb893 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:41.540986 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f8fa7-abec-4040-949a-eef6fb67dca5 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:41.550925 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:41.583728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd60fe-abfa-4b8b-8382-60b236b21117 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:41.618129 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921c450-7de9-4352-8820-21740cb1cdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:41.618682 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921c450-7de9-4352-8820-21740cb1cdff None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:41.618951 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921c450-7de9-4352-8820-21740cb1cdff None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:41.619242 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921c450-7de9-4352-8820-21740cb1cdff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:41.629331 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921c450-7de9-4352-8820-21740cb1cdff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:41.630752 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921c450-7de9-4352-8820-21740cb1cdff None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:41.641469 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f8fa7-abec-4040-949a-eef6fb67dca5 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:41.645470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-3921c450-7de9-4352-8820-21740cb1cdff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:41.645470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:41.645470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:41.645470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:41.645470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:41.645470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:41.645470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:41.645470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:41.645470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:41.645470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:41.645470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:41.645470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:41.645470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:41.645470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:41.645470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:41.645470 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:41.646497 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:41.647560 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:41.647560 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:41.647560 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:41.647560 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 410/821] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:41.652792 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921c450-7de9-4352-8820-21740cb1cdff None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:41.667705 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:41.679965 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13045db-502a-41c7-8855-9bfde1272a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:41.680621 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13045db-502a-41c7-8855-9bfde1272a28 None admin] REQUEST_METHOD: `PUT` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:41.680890 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13045db-502a-41c7-8855-9bfde1272a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:41.681151 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13045db-502a-41c7-8855-9bfde1272a28 None admin] PATH_INFO: `/v3/projects/743463bbeb374c089f42bb822d0fb893/users/fb3d5e300ce94a68a7cee589212a6ddf/roles/4b1f48ab911e4fadbc70164ae43d508b` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:41.683030 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13045db-502a-41c7-8855-9bfde1272a28 None admin] RBAC: Authorizing `identity:create_grant(project_id=743463bbeb374c089f42bb822d0fb893, user_id=fb3d5e300ce94a68a7cee589212a6ddf, role_id=4b1f48ab911e4fadbc70164ae43d508b)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:41.684466 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13045db-502a-41c7-8855-9bfde1272a28 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:41.698381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 412/822] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:27:41 2022] PUT /identity/v3/projects/743463bbeb374c089f42bb822d0fb893/users/fb3d5e300ce94a68a7cee589212a6ddf/roles/4b1f48ab911e4fadbc70164ae43d508b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 14:27:41.758286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921c450-7de9-4352-8820-21740cb1cdff None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:41.794371 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852e23e-1a9a-40b1-af9e-1c8d13474ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:41.794986 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852e23e-1a9a-40b1-af9e-1c8d13474ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:41.795292 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852e23e-1a9a-40b1-af9e-1c8d13474ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:41.795561 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852e23e-1a9a-40b1-af9e-1c8d13474ff7 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:41.796206 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852e23e-1a9a-40b1-af9e-1c8d13474ff7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:41.796504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852e23e-1a9a-40b1-af9e-1c8d13474ff7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:41.797882 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852e23e-1a9a-40b1-af9e-1c8d13474ff7 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:41.810683 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 411/823] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:27:41.816691 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13045db-502a-41c7-8855-9bfde1272a28 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:41.886628 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:41.921084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13045db-502a-41c7-8855-9bfde1272a28 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:41.953950 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d96461-2964-4ebb-8420-960974006c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:41.954573 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d96461-2964-4ebb-8420-960974006c99 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:41.954842 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d96461-2964-4ebb-8420-960974006c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:41.955105 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d96461-2964-4ebb-8420-960974006c99 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:41.962096 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d96461-2964-4ebb-8420-960974006c99 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:41.963554 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d96461-2964-4ebb-8420-960974006c99 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:41.974784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-f4d96461-2964-4ebb-8420-960974006c99 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 14:27:41.974784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:41.974784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:41.974784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:41.974784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:41.974784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:41.974784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:41.974784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:41.974784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:41.974784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:41.974784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:41.974784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:41.974784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:27:41.974784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:27:41.974784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:27:41.974784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:41.976434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:41.977369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:41.977369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:41.977369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:27:41.977369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:27:41.977369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:27:41.977369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:27:41.977369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:27:41.977369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:27:41.977369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 14:27:41.977369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:41.977369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 413/824] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Oct 12 14:27:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:41.982181 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852e23e-1a9a-40b1-af9e-1c8d13474ff7 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:42.050760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:42.080266 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852e23e-1a9a-40b1-af9e-1c8d13474ff7 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:42.112547 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae953c-f769-4b1b-bb7a-bc6441d00a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:42.113149 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae953c-f769-4b1b-bb7a-bc6441d00a4f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:42.113423 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae953c-f769-4b1b-bb7a-bc6441d00a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:42.113689 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae953c-f769-4b1b-bb7a-bc6441d00a4f None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:42.114401 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae953c-f769-4b1b-bb7a-bc6441d00a4f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:42.114700 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae953c-f769-4b1b-bb7a-bc6441d00a4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:42.116641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae953c-f769-4b1b-bb7a-bc6441d00a4f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:42.133570 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 412/825] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Oct 12 14:27:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:42.139439 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d96461-2964-4ebb-8420-960974006c99 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:42.236317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d96461-2964-4ebb-8420-960974006c99 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:42.262390 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:42.270360 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38425734-70d5-4cc0-a91f-77dd7cfc666f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:42.270958 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38425734-70d5-4cc0-a91f-77dd7cfc666f None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:42.271260 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38425734-70d5-4cc0-a91f-77dd7cfc666f None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:42.271528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38425734-70d5-4cc0-a91f-77dd7cfc666f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:42.336218 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38425734-70d5-4cc0-a91f-77dd7cfc666f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:42.337665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38425734-70d5-4cc0-a91f-77dd7cfc666f None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:42.433573 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 414/826] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:27:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 14:27:42.439396 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae953c-f769-4b1b-bb7a-bc6441d00a4f None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:42.534688 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae953c-f769-4b1b-bb7a-bc6441d00a4f None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:42.563154 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:42.570154 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa1585-3f97-486f-aa82-912afb299679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:42.570816 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa1585-3f97-486f-aa82-912afb299679 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:42.571089 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa1585-3f97-486f-aa82-912afb299679 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:42.571382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa1585-3f97-486f-aa82-912afb299679 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:42.584624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa1585-3f97-486f-aa82-912afb299679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:42.586787 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa1585-3f97-486f-aa82-912afb299679 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:42.601565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-67aa1585-3f97-486f-aa82-912afb299679 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:42.601565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:42.601565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:42.601565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:42.601565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:42.601565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:42.601565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:42.601565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:42.601565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:42.601565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:42.601565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:42.601565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:42.601565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:27:42.601565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:27:42.601565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:27:42.601565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:42.602480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:27:42.603394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:27:42.603394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:27:42.603394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:42.603394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 413/827] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:42.608331 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38425734-70d5-4cc0-a91f-77dd7cfc666f None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:42.634937 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:42.703648 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38425734-70d5-4cc0-a91f-77dd7cfc666f None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:42.736480 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161fc02-ffb0-4bae-8290-f72245da9a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:42.737068 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161fc02-ffb0-4bae-8290-f72245da9a96 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:42.737346 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161fc02-ffb0-4bae-8290-f72245da9a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:42.737613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161fc02-ffb0-4bae-8290-f72245da9a96 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:42.738240 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161fc02-ffb0-4bae-8290-f72245da9a96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:42.738535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161fc02-ffb0-4bae-8290-f72245da9a96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:42.739907 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161fc02-ffb0-4bae-8290-f72245da9a96 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:42.751352 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 415/828] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:42.756951 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa1585-3f97-486f-aa82-912afb299679 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:42.855740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa1585-3f97-486f-aa82-912afb299679 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:42.887600 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd651fe8-f7ec-44ec-909b-c099ec85b8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:42.888227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd651fe8-f7ec-44ec-909b-c099ec85b8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:42.888502 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd651fe8-f7ec-44ec-909b-c099ec85b8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:42.888769 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd651fe8-f7ec-44ec-909b-c099ec85b8c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:42.895167 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd651fe8-f7ec-44ec-909b-c099ec85b8c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:42.896598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd651fe8-f7ec-44ec-909b-c099ec85b8c9 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:42.908647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-cd651fe8-f7ec-44ec-909b-c099ec85b8c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:42.908647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:42.908647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:42.908647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:42.908647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:42.908647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:42.908647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:42.908647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:42.908647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:42.908647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:42.908647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:42.908647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:42.908647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:27:42.908647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:27:42.908647 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:42.909641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:42.910499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:27:42.910499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:27:42.910499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:27:42.910499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:27:42.910499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:27:42.910499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:27:42.910499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:27:42.910499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:27:42.910499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 414/829] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:27:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:27:42.916107 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161fc02-ffb0-4bae-8290-f72245da9a96 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:43.001241 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:43.013258 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161fc02-ffb0-4bae-8290-f72245da9a96 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:43.046440 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa66cb-f37b-4ab7-bbf9-4990fd9062d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:43.047037 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa66cb-f37b-4ab7-bbf9-4990fd9062d9 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:43.047339 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa66cb-f37b-4ab7-bbf9-4990fd9062d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:43.047608 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa66cb-f37b-4ab7-bbf9-4990fd9062d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:43.048283 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffa66cb-f37b-4ab7-bbf9-4990fd9062d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:43.048581 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffa66cb-f37b-4ab7-bbf9-4990fd9062d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:43.049989 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffa66cb-f37b-4ab7-bbf9-4990fd9062d9 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:43.061810 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 416/830] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:27:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:43.067660 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd651fe8-f7ec-44ec-909b-c099ec85b8c9 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:43.109716 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:43.164249 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd651fe8-f7ec-44ec-909b-c099ec85b8c9 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:43.197392 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28020c2-ba77-4545-b9dc-cadfa7dbe6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:43.197994 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28020c2-ba77-4545-b9dc-cadfa7dbe6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:43.198270 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28020c2-ba77-4545-b9dc-cadfa7dbe6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:43.198538 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28020c2-ba77-4545-b9dc-cadfa7dbe6b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:43.199285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28020c2-ba77-4545-b9dc-cadfa7dbe6b2 None admin] RBAC: Adding query filter params (role.id=f074c279fe6544eba0c9d54f5150fbd6, scope.project.id=720b11237d2e48128c171be45fcd0ad0, user.id=d0aeed5f32a1477dbd9d3e934669705b) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:43.199590 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28020c2-ba77-4545-b9dc-cadfa7dbe6b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:43.201074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28020c2-ba77-4545-b9dc-cadfa7dbe6b2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:43.206911 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 415/831] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:27:43 2022] GET /identity/v3/role_assignments?user.id=d0aeed5f32a1477dbd9d3e934669705b&scope.project.id=720b11237d2e48128c171be45fcd0ad0&role.id=f074c279fe6544eba0c9d54f5150fbd6 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:43.838659 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681fdf0-d7f2-419e-ac24-5125cc8881b2 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:43.839180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681fdf0-d7f2-419e-ac24-5125cc8881b2 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:43.839180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681fdf0-d7f2-419e-ac24-5125cc8881b2 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:43.840537 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 417/832] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:43.845897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe612c-f6bb-454f-89a3-45ae1a77779e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:43.846305 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe612c-f6bb-454f-89a3-45ae1a77779e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:43.846568 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe612c-f6bb-454f-89a3-45ae1a77779e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:43.865347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c4fe612c-f6bb-454f-89a3-45ae1a77779e None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:43.867215 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c4fe612c-f6bb-454f-89a3-45ae1a77779e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:43.935313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 416/833] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:44.198907 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681fdf0-d7f2-419e-ac24-5125cc8881b2 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:44.295615 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681fdf0-d7f2-419e-ac24-5125cc8881b2 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:44.328188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4d632-6617-462c-924c-337309d11387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:44.328798 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4d632-6617-462c-924c-337309d11387 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:44.329068 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4d632-6617-462c-924c-337309d11387 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:44.329333 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4d632-6617-462c-924c-337309d11387 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:44.339515 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4d632-6617-462c-924c-337309d11387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:27:44.344295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-16f4d632-6617-462c-924c-337309d11387 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:27:44.344295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:27:44.344295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:27:44.344295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:27:44.344295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:27:44.344295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:27:44.344295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:27:44.344295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:27:44.344295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:27:44.344295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:27:44.344295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:27:44.344295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:27:44.344295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:27:44.344295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:27:44.344295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:27:44.344295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:27:44.345371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:27:44.345371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:27:44.345371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:27:44.345371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:27:44.345371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:27:44.345371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:27:44.345371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:27:44.345371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:27:44.345371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:27:44.345371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:27:44.345371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:27:44.345371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:27:44.345371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:27:44.345371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:27:44.345371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:27:44.345371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:27:44.346426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:27:44.346426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:27:44.346426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:27:44.346426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:27:44.346426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:27:44.346426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:27:44.346426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:27:44.346426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:27:44.346426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:27:44.346426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:27:44.346426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:27:44.346426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:27:44.346426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 418/834] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:27:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:27:44.352006 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe612c-f6bb-454f-89a3-45ae1a77779e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:44.384711 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:44.447313 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe612c-f6bb-454f-89a3-45ae1a77779e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:44.479670 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5740fca-0029-4e21-9e96-0b849bcc8fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:44.480204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5740fca-0029-4e21-9e96-0b849bcc8fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:44.480479 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5740fca-0029-4e21-9e96-0b849bcc8fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:44.480749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5740fca-0029-4e21-9e96-0b849bcc8fc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:44.481399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5740fca-0029-4e21-9e96-0b849bcc8fc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:44.481697 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5740fca-0029-4e21-9e96-0b849bcc8fc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:44.483048 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5740fca-0029-4e21-9e96-0b849bcc8fc0 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:44.490728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 417/835] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:27:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:44.497275 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4d632-6617-462c-924c-337309d11387 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:44.508854 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:44.594316 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4d632-6617-462c-924c-337309d11387 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:44.627823 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa45021-a9e6-4fb0-aa50-606c1c95361a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:44.628435 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa45021-a9e6-4fb0-aa50-606c1c95361a None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:44.628710 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa45021-a9e6-4fb0-aa50-606c1c95361a None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:44.628978 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa45021-a9e6-4fb0-aa50-606c1c95361a None admin] PATH_INFO: `/v3/users/fb3d5e300ce94a68a7cee589212a6ddf` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:44.630081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa45021-a9e6-4fb0-aa50-606c1c95361a None admin] RBAC: Authorizing `identity:get_user(user_id=fb3d5e300ce94a68a7cee589212a6ddf)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:44.631515 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa45021-a9e6-4fb0-aa50-606c1c95361a None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:44.633289 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 419/836] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:27:44 2022] GET /identity/v3/users/fb3d5e300ce94a68a7cee589212a6ddf => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:44.638993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5740fca-0029-4e21-9e96-0b849bcc8fc0 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:44.730720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:44.734616 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5740fca-0029-4e21-9e96-0b849bcc8fc0 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:44.766942 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5791d1-88c1-48f8-8a66-8cd20c3b6d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:44.767562 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5791d1-88c1-48f8-8a66-8cd20c3b6d8d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:44.767838 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5791d1-88c1-48f8-8a66-8cd20c3b6d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:44.768105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5791d1-88c1-48f8-8a66-8cd20c3b6d8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:44.832630 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5791d1-88c1-48f8-8a66-8cd20c3b6d8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:44.834047 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5791d1-88c1-48f8-8a66-8cd20c3b6d8d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:44.923742 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:44.928159 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 418/837] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:27:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 14:27:44.933937 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa45021-a9e6-4fb0-aa50-606c1c95361a None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:45.007308 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:45.029807 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa45021-a9e6-4fb0-aa50-606c1c95361a None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:45.061622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07ba11-3399-49d2-ab5d-fb764c5f1843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:45.062225 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07ba11-3399-49d2-ab5d-fb764c5f1843 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:45.062494 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07ba11-3399-49d2-ab5d-fb764c5f1843 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:45.062757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07ba11-3399-49d2-ab5d-fb764c5f1843 None admin] PATH_INFO: `/v3/projects/743463bbeb374c089f42bb822d0fb893` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:45.063855 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07ba11-3399-49d2-ab5d-fb764c5f1843 None admin] RBAC: Authorizing `identity:get_project(project_id=743463bbeb374c089f42bb822d0fb893)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:45.065257 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07ba11-3399-49d2-ab5d-fb764c5f1843 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:45.066830 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 420/838] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:27:44 2022] GET /identity/v3/projects/743463bbeb374c089f42bb822d0fb893 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:45.073017 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5791d1-88c1-48f8-8a66-8cd20c3b6d8d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:45.167692 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5791d1-88c1-48f8-8a66-8cd20c3b6d8d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:45.199613 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1b95b-0ebe-4d05-8afc-16d3000f4ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:45.200200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1b95b-0ebe-4d05-8afc-16d3000f4ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:45.200474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1b95b-0ebe-4d05-8afc-16d3000f4ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:45.200744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1b95b-0ebe-4d05-8afc-16d3000f4ef9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:45.201822 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1b95b-0ebe-4d05-8afc-16d3000f4ef9 None admin] RBAC: Adding query filter params (role.id=4b1f48ab911e4fadbc70164ae43d508b, scope.project.id=743463bbeb374c089f42bb822d0fb893, user.id=fb3d5e300ce94a68a7cee589212a6ddf) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:45.202123 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1b95b-0ebe-4d05-8afc-16d3000f4ef9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:45.203645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1b95b-0ebe-4d05-8afc-16d3000f4ef9 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:45.209328 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 419/839] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:27:45 2022] GET /identity/v3/role_assignments?user.id=fb3d5e300ce94a68a7cee589212a6ddf&scope.project.id=743463bbeb374c089f42bb822d0fb893&role.id=4b1f48ab911e4fadbc70164ae43d508b => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:47.334494 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3213061-9e47-4659-b23a-434c60eb8903 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:47.338053 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3213061-9e47-4659-b23a-434c60eb8903 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:47.338053 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3213061-9e47-4659-b23a-434c60eb8903 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:47.338053 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 421/840] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:47.342646 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c459e7-78f3-4e02-a74d-25210264a867 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:47.343052 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c459e7-78f3-4e02-a74d-25210264a867 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:47.343225 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c459e7-78f3-4e02-a74d-25210264a867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:47.361538 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-97c459e7-78f3-4e02-a74d-25210264a867 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:47.362720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-97c459e7-78f3-4e02-a74d-25210264a867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:47.416674 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:47.426916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 420/841] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:47.696766 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3213061-9e47-4659-b23a-434c60eb8903 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:47.794131 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3213061-9e47-4659-b23a-434c60eb8903 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:47.826917 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381df6e-e94a-4798-80a8-7d52d4c5330c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:47.827623 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381df6e-e94a-4798-80a8-7d52d4c5330c None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:47.827903 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381df6e-e94a-4798-80a8-7d52d4c5330c None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:47.828169 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381df6e-e94a-4798-80a8-7d52d4c5330c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:47.829261 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381df6e-e94a-4798-80a8-7d52d4c5330c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:47.830670 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381df6e-e94a-4798-80a8-7d52d4c5330c None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:47.832216 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 422/842] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:47.838015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c459e7-78f3-4e02-a74d-25210264a867 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:47.934366 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c459e7-78f3-4e02-a74d-25210264a867 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:47.966390 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6d7df-da69-483c-9225-83274554eaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:47.966924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6d7df-da69-483c-9225-83274554eaa2 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:47.967217 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6d7df-da69-483c-9225-83274554eaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:47.967484 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6d7df-da69-483c-9225-83274554eaa2 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:47.968127 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6d7df-da69-483c-9225-83274554eaa2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:47.969530 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6d7df-da69-483c-9225-83274554eaa2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:47.990519 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 421/843] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:27:47 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:27:50.142731 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cd23c-9f9b-435b-957b-07c6cd14bd93 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:50.143319 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cd23c-9f9b-435b-957b-07c6cd14bd93 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:50.143319 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cd23c-9f9b-435b-957b-07c6cd14bd93 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:50.144652 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 423/844] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:50.150080 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f0c29-195c-4da7-afd9-3a03e5e32ebc None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:50.150499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f0c29-195c-4da7-afd9-3a03e5e32ebc None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:50.150626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f0c29-195c-4da7-afd9-3a03e5e32ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:50.169093 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-362f0c29-195c-4da7-afd9-3a03e5e32ebc None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:50.170269 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-362f0c29-195c-4da7-afd9-3a03e5e32ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:50.211850 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:50.235008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 422/845] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:50.506181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cd23c-9f9b-435b-957b-07c6cd14bd93 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:50.520531 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:50.605840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cd23c-9f9b-435b-957b-07c6cd14bd93 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:50.637971 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4939f-c725-49b5-84f9-67ee719d24e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:50.638587 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4939f-c725-49b5-84f9-67ee719d24e3 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:50.638859 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4939f-c725-49b5-84f9-67ee719d24e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:50.639126 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4939f-c725-49b5-84f9-67ee719d24e3 None admin] PATH_INFO: `/v3/roles/f0761dd38b794ee6bfab00ac01f240cd` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:50.640517 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4939f-c725-49b5-84f9-67ee719d24e3 None admin] RBAC: Authorizing `identity:get_role(role_id=f0761dd38b794ee6bfab00ac01f240cd)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:50.641866 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4939f-c725-49b5-84f9-67ee719d24e3 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:50.643183 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 424/846] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:27:50 2022] GET /identity/v3/roles/f0761dd38b794ee6bfab00ac01f240cd => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:50.649769 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f0c29-195c-4da7-afd9-3a03e5e32ebc None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:50.746326 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f0c29-195c-4da7-afd9-3a03e5e32ebc None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:50.778191 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2bd31-f146-4063-bc73-f80204616236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:50.778749 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2bd31-f146-4063-bc73-f80204616236 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:50.779022 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2bd31-f146-4063-bc73-f80204616236 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:50.779321 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2bd31-f146-4063-bc73-f80204616236 None admin] PATH_INFO: `/v3/users/33dba583f8784f378a2cfdf273888564` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:50.792130 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2bd31-f146-4063-bc73-f80204616236 None admin] RBAC: Authorizing `identity:get_user(user_id=33dba583f8784f378a2cfdf273888564)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:50.793550 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2bd31-f146-4063-bc73-f80204616236 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:50.795099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 423/847] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:27:50 2022] GET /identity/v3/users/33dba583f8784f378a2cfdf273888564 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:50.800814 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4939f-c725-49b5-84f9-67ee719d24e3 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:50.877515 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:50.896017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4939f-c725-49b5-84f9-67ee719d24e3 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:50.928171 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8ed8b-edbd-4d55-85ce-3c6a574a0db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:50.928771 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8ed8b-edbd-4d55-85ce-3c6a574a0db7 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:50.929166 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8ed8b-edbd-4d55-85ce-3c6a574a0db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:50.929442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8ed8b-edbd-4d55-85ce-3c6a574a0db7 None admin] PATH_INFO: `/v3/projects/743463bbeb374c089f42bb822d0fb893` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:50.930536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8ed8b-edbd-4d55-85ce-3c6a574a0db7 None admin] RBAC: Authorizing `identity:get_project(project_id=743463bbeb374c089f42bb822d0fb893)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:50.931965 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8ed8b-edbd-4d55-85ce-3c6a574a0db7 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:50.933554 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 425/848] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:27:50 2022] GET /identity/v3/projects/743463bbeb374c089f42bb822d0fb893 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:50.939466 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2bd31-f146-4063-bc73-f80204616236 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:50.973791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:51.035825 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2bd31-f146-4063-bc73-f80204616236 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:51.068206 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2b6fd-3e6f-4ee7-b227-4f6e816d5766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:51.068879 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2b6fd-3e6f-4ee7-b227-4f6e816d5766 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:51.069228 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2b6fd-3e6f-4ee7-b227-4f6e816d5766 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:51.069499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2b6fd-3e6f-4ee7-b227-4f6e816d5766 None admin] PATH_INFO: `/v3/projects/743463bbeb374c089f42bb822d0fb893/users/33dba583f8784f378a2cfdf273888564/roles/f0761dd38b794ee6bfab00ac01f240cd` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:51.072323 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2b6fd-3e6f-4ee7-b227-4f6e816d5766 None admin] RBAC: Authorizing `identity:create_grant(project_id=743463bbeb374c089f42bb822d0fb893, user_id=33dba583f8784f378a2cfdf273888564, role_id=f0761dd38b794ee6bfab00ac01f240cd)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:51.073742 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2b6fd-3e6f-4ee7-b227-4f6e816d5766 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:51.086151 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 424/849] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:27:50 2022] PUT /identity/v3/projects/743463bbeb374c089f42bb822d0fb893/users/33dba583f8784f378a2cfdf273888564/roles/f0761dd38b794ee6bfab00ac01f240cd => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:27:53.195568 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e1fa8-9bb6-44ba-9088-8455f4348d4b None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:53.196283 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e1fa8-9bb6-44ba-9088-8455f4348d4b None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:53.196283 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e1fa8-9bb6-44ba-9088-8455f4348d4b None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:53.197520 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 426/850] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:53.203380 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba36c7-62f7-4edb-9686-b4e3f3174813 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:53.203824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba36c7-62f7-4edb-9686-b4e3f3174813 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:53.204070 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba36c7-62f7-4edb-9686-b4e3f3174813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:53.222769 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-75ba36c7-62f7-4edb-9686-b4e3f3174813 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:53.223955 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-75ba36c7-62f7-4edb-9686-b4e3f3174813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:53.288844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 425/851] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 14:27:53.553144 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e1fa8-9bb6-44ba-9088-8455f4348d4b None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:53.652918 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e1fa8-9bb6-44ba-9088-8455f4348d4b None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:53.685829 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db317ea-6228-4e20-9d50-8208c42f89cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:53.686444 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db317ea-6228-4e20-9d50-8208c42f89cb None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:53.686719 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db317ea-6228-4e20-9d50-8208c42f89cb None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:53.686988 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db317ea-6228-4e20-9d50-8208c42f89cb None admin] PATH_INFO: `/v3/roles/f0761dd38b794ee6bfab00ac01f240cd` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:53.688379 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db317ea-6228-4e20-9d50-8208c42f89cb None admin] RBAC: Authorizing `identity:get_role(role_id=f0761dd38b794ee6bfab00ac01f240cd)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:53.689729 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db317ea-6228-4e20-9d50-8208c42f89cb None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:53.691023 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 427/852] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:27:53 2022] GET /identity/v3/roles/f0761dd38b794ee6bfab00ac01f240cd => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:53.697046 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba36c7-62f7-4edb-9686-b4e3f3174813 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:53.722593 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:53.794961 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba36c7-62f7-4edb-9686-b4e3f3174813 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:53.828312 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfe3ba-0578-4685-87d8-ef9e40827f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:53.828869 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfe3ba-0578-4685-87d8-ef9e40827f2d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:53.829140 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfe3ba-0578-4685-87d8-ef9e40827f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:53.829409 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfe3ba-0578-4685-87d8-ef9e40827f2d None admin] PATH_INFO: `/v3/users/33dba583f8784f378a2cfdf273888564` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:53.830515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfe3ba-0578-4685-87d8-ef9e40827f2d None admin] RBAC: Authorizing `identity:get_user(user_id=33dba583f8784f378a2cfdf273888564)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:53.831955 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfe3ba-0578-4685-87d8-ef9e40827f2d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:53.833504 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 426/853] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:27:53 2022] GET /identity/v3/users/33dba583f8784f378a2cfdf273888564 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:53.839395 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db317ea-6228-4e20-9d50-8208c42f89cb None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:53.857290 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:53.936240 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db317ea-6228-4e20-9d50-8208c42f89cb None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:53.968439 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5e819-f96c-42a0-bd62-3955c72e5253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:53.969054 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5e819-f96c-42a0-bd62-3955c72e5253 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:53.969329 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5e819-f96c-42a0-bd62-3955c72e5253 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:53.969598 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5e819-f96c-42a0-bd62-3955c72e5253 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:54.034401 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5e819-f96c-42a0-bd62-3955c72e5253 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:54.035965 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5e819-f96c-42a0-bd62-3955c72e5253 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:54.052461 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:54.130586 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 428/854] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:27:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 14:27:54.136378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfe3ba-0578-4685-87d8-ef9e40827f2d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:54.225591 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:54.233410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfe3ba-0578-4685-87d8-ef9e40827f2d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:54.266562 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d238c6-0fd3-41f0-818a-f9280efa6a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:54.267203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d238c6-0fd3-41f0-818a-f9280efa6a2a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:54.267476 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d238c6-0fd3-41f0-818a-f9280efa6a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:54.267743 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d238c6-0fd3-41f0-818a-f9280efa6a2a None admin] PATH_INFO: `/v3/projects/743463bbeb374c089f42bb822d0fb893` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:54.268818 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d238c6-0fd3-41f0-818a-f9280efa6a2a None admin] RBAC: Authorizing `identity:get_project(project_id=743463bbeb374c089f42bb822d0fb893)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:54.270226 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d238c6-0fd3-41f0-818a-f9280efa6a2a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:54.271887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 427/855] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:27:54 2022] GET /identity/v3/projects/743463bbeb374c089f42bb822d0fb893 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:54.279253 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5e819-f96c-42a0-bd62-3955c72e5253 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:54.373901 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5e819-f96c-42a0-bd62-3955c72e5253 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:54.390713 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:54.408957 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7a46d-401d-4658-9a5d-cc6c2047b4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:54.409538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7a46d-401d-4658-9a5d-cc6c2047b4bf None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:54.409808 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7a46d-401d-4658-9a5d-cc6c2047b4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:54.410071 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7a46d-401d-4658-9a5d-cc6c2047b4bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:54.410798 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7a46d-401d-4658-9a5d-cc6c2047b4bf None admin] RBAC: Adding query filter params (role.id=f0761dd38b794ee6bfab00ac01f240cd, scope.project.id=743463bbeb374c089f42bb822d0fb893, user.id=33dba583f8784f378a2cfdf273888564) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:27:54.411094 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7a46d-401d-4658-9a5d-cc6c2047b4bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:54.412526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7a46d-401d-4658-9a5d-cc6c2047b4bf None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:54.418423 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 429/856] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:27:54 2022] GET /identity/v3/role_assignments?user.id=33dba583f8784f378a2cfdf273888564&scope.project.id=743463bbeb374c089f42bb822d0fb893&role.id=f0761dd38b794ee6bfab00ac01f240cd => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:56.542487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ba38f-1d1a-40fb-8ee5-880cd3a4315d None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:56.543017 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ba38f-1d1a-40fb-8ee5-880cd3a4315d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:56.543017 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ba38f-1d1a-40fb-8ee5-880cd3a4315d None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:56.544370 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 428/857] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:56.549477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2676a4-840e-4d0c-a400-a1150df66b5f None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:56.549908 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2676a4-840e-4d0c-a400-a1150df66b5f None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:56.550000 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2676a4-840e-4d0c-a400-a1150df66b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:56.568051 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-1b2676a4-840e-4d0c-a400-a1150df66b5f None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:56.569210 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-1b2676a4-840e-4d0c-a400-a1150df66b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:56.634511 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 430/858] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:56.903451 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ba38f-1d1a-40fb-8ee5-880cd3a4315d None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:57.004397 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ba38f-1d1a-40fb-8ee5-880cd3a4315d None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:57.037088 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a658ea-62c8-4c7e-ba92-2d10849d22b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:57.037704 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a658ea-62c8-4c7e-ba92-2d10849d22b5 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:57.037975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a658ea-62c8-4c7e-ba92-2d10849d22b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:57.038240 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a658ea-62c8-4c7e-ba92-2d10849d22b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:57.040237 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a658ea-62c8-4c7e-ba92-2d10849d22b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:57.041640 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a658ea-62c8-4c7e-ba92-2d10849d22b5 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:57.043185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 429/859] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:57.049002 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2676a4-840e-4d0c-a400-a1150df66b5f None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:57.142883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2676a4-840e-4d0c-a400-a1150df66b5f None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:57.153937 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:57.176702 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611ca93-eca3-4c93-93be-fd84866596fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:57.177285 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611ca93-eca3-4c93-93be-fd84866596fa None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:57.177554 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611ca93-eca3-4c93-93be-fd84866596fa None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:57.177818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611ca93-eca3-4c93-93be-fd84866596fa None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:57.178457 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611ca93-eca3-4c93-93be-fd84866596fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:57.180691 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611ca93-eca3-4c93-93be-fd84866596fa None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:57.206348 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 431/860] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Oct 12 14:27:57 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 14:27:59.373565 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ad44b-7f0d-4e84-b696-f0b2cce84c63 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:59.374124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ad44b-7f0d-4e84-b696-f0b2cce84c63 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:59.374124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ad44b-7f0d-4e84-b696-f0b2cce84c63 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:59.375431 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 430/861] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:27:59.380551 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d22f7c-7333-4683-a764-8427ca453acd None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:59.380923 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d22f7c-7333-4683-a764-8427ca453acd None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:59.381084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d22f7c-7333-4683-a764-8427ca453acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:59.399673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-96d22f7c-7333-4683-a764-8427ca453acd None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:27:59.400824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-96d22f7c-7333-4683-a764-8427ca453acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:27:59.464375 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 432/862] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:27:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:27:59.733178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ad44b-7f0d-4e84-b696-f0b2cce84c63 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:59.763132 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:59.832344 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ad44b-7f0d-4e84-b696-f0b2cce84c63 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:27:59.864845 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47097b-0c9f-4868-a68b-c157b2458e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:27:59.865454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47097b-0c9f-4868-a68b-c157b2458e00 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:27:59.865726 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47097b-0c9f-4868-a68b-c157b2458e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:27:59.865990 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47097b-0c9f-4868-a68b-c157b2458e00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:27:59.867752 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47097b-0c9f-4868-a68b-c157b2458e00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:27:59.869336 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47097b-0c9f-4868-a68b-c157b2458e00 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:27:59.870843 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 431/863] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:27:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:27:59.876438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d22f7c-7333-4683-a764-8427ca453acd None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:27:59.962862 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:27:59.972145 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d22f7c-7333-4683-a764-8427ca453acd None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:00.005608 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359365a4-51fe-4a39-8589-0357df50d37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:00.006140 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359365a4-51fe-4a39-8589-0357df50d37c None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:00.006417 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359365a4-51fe-4a39-8589-0357df50d37c None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:00.006685 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359365a4-51fe-4a39-8589-0357df50d37c None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:00.007363 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359365a4-51fe-4a39-8589-0357df50d37c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:00.008972 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359365a4-51fe-4a39-8589-0357df50d37c None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:00.030051 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 433/864] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:27:59 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:28:02.169577 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60273a4-8f5e-469d-95b4-9b91378532be None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:02.170109 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60273a4-8f5e-469d-95b4-9b91378532be None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:02.170109 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60273a4-8f5e-469d-95b4-9b91378532be None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:02.171476 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 432/865] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:28:02.176616 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa196501-a9b2-4fd5-bd08-f58eec168ab7 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:02.177038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa196501-a9b2-4fd5-bd08-f58eec168ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:02.177164 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa196501-a9b2-4fd5-bd08-f58eec168ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:02.194904 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-fa196501-a9b2-4fd5-bd08-f58eec168ab7 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:28:02.196701 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-fa196501-a9b2-4fd5-bd08-f58eec168ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:28:02.261725 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 434/866] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:28:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:28:02.532077 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60273a4-8f5e-469d-95b4-9b91378532be None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:02.625025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:02.632090 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60273a4-8f5e-469d-95b4-9b91378532be None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:02.669065 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89762b0e-7770-4c05-8cbf-6b416199507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:02.669918 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89762b0e-7770-4c05-8cbf-6b416199507f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:02.670194 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89762b0e-7770-4c05-8cbf-6b416199507f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:02.670463 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89762b0e-7770-4c05-8cbf-6b416199507f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:02.680260 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89762b0e-7770-4c05-8cbf-6b416199507f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:28:02.685027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-89762b0e-7770-4c05-8cbf-6b416199507f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:28:02.685027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:28:02.685027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:28:02.685027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:28:02.685027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:28:02.685027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:28:02.685027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:28:02.685027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:28:02.685027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:28:02.685027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:28:02.685027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:28:02.685027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:28:02.685027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:28:02.685027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:28:02.685027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:28:02.685027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:28:02.686185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:28:02.686185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:28:02.686185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:28:02.686185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:28:02.686185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:28:02.686185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:28:02.686185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:28:02.686185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:28:02.686185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:28:02.686185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:28:02.686185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:28:02.686185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:28:02.686185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:28:02.686185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:28:02.686185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:28:02.686185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:28:02.687535 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:28:02.687535 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:28:02.687535 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:28:02.687535 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:28:02.687535 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:28:02.687535 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:28:02.687535 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:28:02.687535 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:28:02.687535 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:28:02.687535 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:28:02.687535 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:28:02.687535 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:28:02.687535 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 433/867] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:28:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:28:02.692119 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa196501-a9b2-4fd5-bd08-f58eec168ab7 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:02.767034 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:02.789602 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa196501-a9b2-4fd5-bd08-f58eec168ab7 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:02.822564 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9057a-4565-4957-95fe-33eea02b5890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:02.823180 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9057a-4565-4957-95fe-33eea02b5890 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:02.823460 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9057a-4565-4957-95fe-33eea02b5890 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:02.823726 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9057a-4565-4957-95fe-33eea02b5890 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:02.824382 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9057a-4565-4957-95fe-33eea02b5890 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:28:02.824677 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9057a-4565-4957-95fe-33eea02b5890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:02.826039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9057a-4565-4957-95fe-33eea02b5890 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:02.833766 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 435/868] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:28:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:28:02.840353 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89762b0e-7770-4c05-8cbf-6b416199507f None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:02.937968 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89762b0e-7770-4c05-8cbf-6b416199507f None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:02.969756 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e51f2d-3ff2-4214-ae2a-577207c972bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:02.970366 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e51f2d-3ff2-4214-ae2a-577207c972bc None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:02.970638 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e51f2d-3ff2-4214-ae2a-577207c972bc None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:02.970908 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e51f2d-3ff2-4214-ae2a-577207c972bc None admin] PATH_INFO: `/v3/users/c9f93a4e1761456eb8e5d71290fe2b34` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:02.984343 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e51f2d-3ff2-4214-ae2a-577207c972bc None admin] RBAC: Authorizing `identity:get_user(user_id=c9f93a4e1761456eb8e5d71290fe2b34)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:02.985762 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e51f2d-3ff2-4214-ae2a-577207c972bc None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:02.987366 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 434/869] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:28:02 2022] GET /identity/v3/users/c9f93a4e1761456eb8e5d71290fe2b34 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:28:02.993785 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9057a-4565-4957-95fe-33eea02b5890 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:03.089999 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9057a-4565-4957-95fe-33eea02b5890 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:03.122558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868e42c-5de1-4553-93f2-580fdfec7b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:03.123189 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868e42c-5de1-4553-93f2-580fdfec7b7b None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:03.123466 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868e42c-5de1-4553-93f2-580fdfec7b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:03.123730 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868e42c-5de1-4553-93f2-580fdfec7b7b None admin] PATH_INFO: `/v3/projects/f1929613921243e6a206e7ccfacd5eee` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:03.124833 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868e42c-5de1-4553-93f2-580fdfec7b7b None admin] RBAC: Authorizing `identity:get_project(project_id=f1929613921243e6a206e7ccfacd5eee)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:03.126245 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868e42c-5de1-4553-93f2-580fdfec7b7b None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:03.127876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 436/870] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:28:02 2022] GET /identity/v3/projects/f1929613921243e6a206e7ccfacd5eee => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:28:03.133946 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e51f2d-3ff2-4214-ae2a-577207c972bc None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:03.151752 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:03.229306 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e51f2d-3ff2-4214-ae2a-577207c972bc None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:03.261377 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffdf11-79fa-4a1b-a936-a2d6cb14ac20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:03.262007 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffdf11-79fa-4a1b-a936-a2d6cb14ac20 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:03.262280 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffdf11-79fa-4a1b-a936-a2d6cb14ac20 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:03.262547 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffdf11-79fa-4a1b-a936-a2d6cb14ac20 None admin] PATH_INFO: `/v3/projects/f1929613921243e6a206e7ccfacd5eee/users/c9f93a4e1761456eb8e5d71290fe2b34/roles/4b1f48ab911e4fadbc70164ae43d508b` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:03.264415 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbffdf11-79fa-4a1b-a936-a2d6cb14ac20 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1929613921243e6a206e7ccfacd5eee, user_id=c9f93a4e1761456eb8e5d71290fe2b34, role_id=4b1f48ab911e4fadbc70164ae43d508b)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:03.265808 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbffdf11-79fa-4a1b-a936-a2d6cb14ac20 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:03.278385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 435/871] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:28:03 2022] PUT /identity/v3/projects/f1929613921243e6a206e7ccfacd5eee/users/c9f93a4e1761456eb8e5d71290fe2b34/roles/4b1f48ab911e4fadbc70164ae43d508b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 14:28:05.380684 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eab174-d7f0-4da1-a4b8-b24a4d4a4c60 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:05.381231 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eab174-d7f0-4da1-a4b8-b24a4d4a4c60 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:05.381231 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eab174-d7f0-4da1-a4b8-b24a4d4a4c60 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:05.382539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 437/872] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:28:05.387883 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf780d4-bc57-467a-a7b6-b310a6a78ccf None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:05.388304 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf780d4-bc57-467a-a7b6-b310a6a78ccf None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:05.388411 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf780d4-bc57-467a-a7b6-b310a6a78ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:05.406875 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-daf780d4-bc57-467a-a7b6-b310a6a78ccf None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:28:05.408083 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-daf780d4-bc57-467a-a7b6-b310a6a78ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:28:05.471204 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 436/873] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:28:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:28:05.738147 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eab174-d7f0-4da1-a4b8-b24a4d4a4c60 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:05.749569 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:05.836189 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eab174-d7f0-4da1-a4b8-b24a4d4a4c60 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:05.868434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01203c81-3d7e-480f-bf1f-947a5c8f559e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:05.869038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01203c81-3d7e-480f-bf1f-947a5c8f559e None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:05.869311 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01203c81-3d7e-480f-bf1f-947a5c8f559e None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:05.869574 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01203c81-3d7e-480f-bf1f-947a5c8f559e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:05.879536 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01203c81-3d7e-480f-bf1f-947a5c8f559e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:28:05.884410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-01203c81-3d7e-480f-bf1f-947a5c8f559e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:28:05.884410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:28:05.884410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:28:05.884410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:28:05.884410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:28:05.884410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:28:05.884410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:28:05.884410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:28:05.884410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:28:05.884410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:28:05.884410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:28:05.884410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:28:05.884410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:28:05.884410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:28:05.884410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:28:05.884410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:28:05.885493 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:28:05.885493 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:28:05.885493 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:28:05.885493 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:28:05.885493 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:28:05.885493 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:28:05.885493 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:28:05.885493 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:28:05.885493 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:28:05.885493 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:28:05.885493 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:28:05.885493 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:28:05.885493 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:28:05.885493 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:28:05.885493 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:28:05.885493 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:28:05.886445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:28:05.886445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:28:05.886445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:28:05.886445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:28:05.886445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:28:05.886445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:28:05.886445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:28:05.886445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:28:05.886445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:28:05.886445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:28:05.886445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:28:05.886445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:28:05.886445 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 438/874] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:28:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:28:05.891456 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf780d4-bc57-467a-a7b6-b310a6a78ccf None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:05.899465 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:05.987650 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf780d4-bc57-467a-a7b6-b310a6a78ccf None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:06.019906 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e1874-c11b-48e8-be3e-36b3a362e551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:06.020438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e1874-c11b-48e8-be3e-36b3a362e551 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:06.020708 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e1874-c11b-48e8-be3e-36b3a362e551 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:06.020970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e1874-c11b-48e8-be3e-36b3a362e551 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:06.021613 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e1874-c11b-48e8-be3e-36b3a362e551 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:28:06.021907 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e1874-c11b-48e8-be3e-36b3a362e551 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:06.023270 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e1874-c11b-48e8-be3e-36b3a362e551 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:06.031620 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 437/875] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:28:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:28:06.037644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01203c81-3d7e-480f-bf1f-947a5c8f559e None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:06.102433 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:06.135220 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01203c81-3d7e-480f-bf1f-947a5c8f559e None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:06.168157 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f435c-09a0-45d2-8bc7-9d1913814a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:06.168762 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f435c-09a0-45d2-8bc7-9d1913814a6f None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:06.169033 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f435c-09a0-45d2-8bc7-9d1913814a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:06.169297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f435c-09a0-45d2-8bc7-9d1913814a6f None admin] PATH_INFO: `/v3/users/c9f93a4e1761456eb8e5d71290fe2b34` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:06.170367 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f435c-09a0-45d2-8bc7-9d1913814a6f None admin] RBAC: Authorizing `identity:get_user(user_id=c9f93a4e1761456eb8e5d71290fe2b34)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:06.171804 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f435c-09a0-45d2-8bc7-9d1913814a6f None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:06.173327 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 439/876] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:28:06 2022] GET /identity/v3/users/c9f93a4e1761456eb8e5d71290fe2b34 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:28:06.179081 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e1874-c11b-48e8-be3e-36b3a362e551 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:06.244139 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:06.276485 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e1874-c11b-48e8-be3e-36b3a362e551 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:06.308453 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0f32f-d60a-4e45-af37-e241de95dbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:06.309045 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0f32f-d60a-4e45-af37-e241de95dbc1 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:06.309324 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0f32f-d60a-4e45-af37-e241de95dbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:06.309593 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0f32f-d60a-4e45-af37-e241de95dbc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:06.372977 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0f32f-d60a-4e45-af37-e241de95dbc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:06.374391 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0f32f-d60a-4e45-af37-e241de95dbc1 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:06.435284 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:06.467934 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 438/877] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:28:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 14:28:06.473776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f435c-09a0-45d2-8bc7-9d1913814a6f None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:06.569005 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f435c-09a0-45d2-8bc7-9d1913814a6f None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:06.603120 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d54069-155c-47a0-889e-281003f37cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:06.603763 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d54069-155c-47a0-889e-281003f37cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:06.604035 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d54069-155c-47a0-889e-281003f37cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:06.604301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d54069-155c-47a0-889e-281003f37cb2 None admin] PATH_INFO: `/v3/projects/f1929613921243e6a206e7ccfacd5eee` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:06.605392 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d54069-155c-47a0-889e-281003f37cb2 None admin] RBAC: Authorizing `identity:get_project(project_id=f1929613921243e6a206e7ccfacd5eee)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:06.606833 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d54069-155c-47a0-889e-281003f37cb2 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:06.608492 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 440/878] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:28:06 2022] GET /identity/v3/projects/f1929613921243e6a206e7ccfacd5eee => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:28:06.614509 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0f32f-d60a-4e45-af37-e241de95dbc1 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:06.710248 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0f32f-d60a-4e45-af37-e241de95dbc1 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:06.743256 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c026931-1d7b-469e-8346-9608e118ee7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:06.743842 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c026931-1d7b-469e-8346-9608e118ee7f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:06.744114 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c026931-1d7b-469e-8346-9608e118ee7f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:06.744380 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c026931-1d7b-469e-8346-9608e118ee7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:06.745086 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c026931-1d7b-469e-8346-9608e118ee7f None admin] RBAC: Adding query filter params (role.id=4b1f48ab911e4fadbc70164ae43d508b, scope.project.id=f1929613921243e6a206e7ccfacd5eee, user.id=c9f93a4e1761456eb8e5d71290fe2b34) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:28:06.745384 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c026931-1d7b-469e-8346-9608e118ee7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:06.746784 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c026931-1d7b-469e-8346-9608e118ee7f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:06.752234 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 439/879] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:28:06 2022] GET /identity/v3/role_assignments?user.id=c9f93a4e1761456eb8e5d71290fe2b34&scope.project.id=f1929613921243e6a206e7ccfacd5eee&role.id=4b1f48ab911e4fadbc70164ae43d508b => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:28:08.882537 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bc9ed-b66e-48e7-831a-5853ea0a0e61 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:08.883701 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bc9ed-b66e-48e7-831a-5853ea0a0e61 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:08.883701 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bc9ed-b66e-48e7-831a-5853ea0a0e61 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:08.885008 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 441/880] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:28:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:28:08.890273 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e51eec-becb-46cc-a2d9-78c9504d8d34 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:08.890675 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e51eec-becb-46cc-a2d9-78c9504d8d34 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:08.890819 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e51eec-becb-46cc-a2d9-78c9504d8d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:08.909481 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-02e51eec-becb-46cc-a2d9-78c9504d8d34 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:28:08.910664 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-02e51eec-becb-46cc-a2d9-78c9504d8d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:28:08.935500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:08.975609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 440/881] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:28:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:28:09.247270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bc9ed-b66e-48e7-831a-5853ea0a0e61 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:09.251032 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:09.350386 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bc9ed-b66e-48e7-831a-5853ea0a0e61 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:09.383385 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87e4b1-f1b4-4e72-9320-267f99d02011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:09.384050 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87e4b1-f1b4-4e72-9320-267f99d02011 None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:09.384323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87e4b1-f1b4-4e72-9320-267f99d02011 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:09.384591 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87e4b1-f1b4-4e72-9320-267f99d02011 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:09.385264 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87e4b1-f1b4-4e72-9320-267f99d02011 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:09.386622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87e4b1-f1b4-4e72-9320-267f99d02011 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:09.402278 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 442/882] 10.222.0.16 () {60 vars in 1138 bytes} [Wed Oct 12 14:28:09 2022] POST /identity/v3/domains => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:28:11.544150 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92293203-3078-4234-b904-8366eb4541dd None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:11.544710 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92293203-3078-4234-b904-8366eb4541dd None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:11.544710 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92293203-3078-4234-b904-8366eb4541dd None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:11.546018 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 441/883] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:28:11.551415 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478000f-5f57-48a5-9289-9d7bc12959b9 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:11.551793 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478000f-5f57-48a5-9289-9d7bc12959b9 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:11.551963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478000f-5f57-48a5-9289-9d7bc12959b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:11.572057 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-9478000f-5f57-48a5-9289-9d7bc12959b9 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:28:11.573240 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-9478000f-5f57-48a5-9289-9d7bc12959b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:28:11.630739 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:11.639800 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 443/884] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:28:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 14:28:11.907499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92293203-3078-4234-b904-8366eb4541dd None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:12.004596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92293203-3078-4234-b904-8366eb4541dd None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:12.036975 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da3b30-c162-4688-af69-ceae0e1836a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:12.037594 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da3b30-c162-4688-af69-ceae0e1836a3 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:12.037868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da3b30-c162-4688-af69-ceae0e1836a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:12.038133 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da3b30-c162-4688-af69-ceae0e1836a3 None admin] PATH_INFO: `/v3/domains/9bbcd623d9c1482ab54d95b30cf833e0` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:12.048638 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da3b30-c162-4688-af69-ceae0e1836a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9bbcd623d9c1482ab54d95b30cf833e0)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:12.050064 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da3b30-c162-4688-af69-ceae0e1836a3 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:12.051640 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 442/885] 10.222.0.16 () {56 vars in 1183 bytes} [Wed Oct 12 14:28:11 2022] GET /identity/v3/domains/9bbcd623d9c1482ab54d95b30cf833e0 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:28:12.057720 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478000f-5f57-48a5-9289-9d7bc12959b9 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:12.151725 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478000f-5f57-48a5-9289-9d7bc12959b9 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:12.185474 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bff41-b658-4349-82a1-f36508bab48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:12.186001 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bff41-b658-4349-82a1-f36508bab48f None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:12.186269 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bff41-b658-4349-82a1-f36508bab48f None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:12.186533 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bff41-b658-4349-82a1-f36508bab48f None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:12.187182 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bff41-b658-4349-82a1-f36508bab48f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:12.188620 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bff41-b658-4349-82a1-f36508bab48f None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:12.214634 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 444/886] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Oct 12 14:28:12 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:28:14.370665 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb306889-ac95-453d-9a4f-4c764cf8804d None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:14.371218 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb306889-ac95-453d-9a4f-4c764cf8804d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:14.371218 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb306889-ac95-453d-9a4f-4c764cf8804d None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:14.372563 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 443/887] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:28:14.377863 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869dec9-6328-48fd-863a-a5cd0d9f703d None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:14.378289 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869dec9-6328-48fd-863a-a5cd0d9f703d None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:14.378389 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869dec9-6328-48fd-863a-a5cd0d9f703d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:14.396764 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-0869dec9-6328-48fd-863a-a5cd0d9f703d None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:28:14.397928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-0869dec9-6328-48fd-863a-a5cd0d9f703d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:28:14.438722 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:14.463841 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 445/888] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:28:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:28:14.729276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb306889-ac95-453d-9a4f-4c764cf8804d None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:14.753485 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:14.828608 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb306889-ac95-453d-9a4f-4c764cf8804d None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:14.860575 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea892a13-fd55-4781-a193-246fe332e680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:14.861189 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea892a13-fd55-4781-a193-246fe332e680 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:14.861465 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea892a13-fd55-4781-a193-246fe332e680 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:14.861735 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea892a13-fd55-4781-a193-246fe332e680 None admin] PATH_INFO: `/v3/domains/9bbcd623d9c1482ab54d95b30cf833e0` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:14.862839 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea892a13-fd55-4781-a193-246fe332e680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9bbcd623d9c1482ab54d95b30cf833e0)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:14.864286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea892a13-fd55-4781-a193-246fe332e680 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:14.865817 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 444/889] 10.222.0.16 () {56 vars in 1183 bytes} [Wed Oct 12 14:28:14 2022] GET /identity/v3/domains/9bbcd623d9c1482ab54d95b30cf833e0 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:28:14.871659 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869dec9-6328-48fd-863a-a5cd0d9f703d None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:14.966626 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869dec9-6328-48fd-863a-a5cd0d9f703d None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:14.999211 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cec29-ed83-453f-a247-6c6d6a627fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:14.999749 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cec29-ed83-453f-a247-6c6d6a627fe0 None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:15.000021 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cec29-ed83-453f-a247-6c6d6a627fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:15.000283 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cec29-ed83-453f-a247-6c6d6a627fe0 None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:15.000928 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cec29-ed83-453f-a247-6c6d6a627fe0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:15.002340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cec29-ed83-453f-a247-6c6d6a627fe0 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:15.022757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 446/890] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:28:14 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:28:17.184128 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fadd85-9b64-4eea-adba-d61189fa3b9b None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:17.184662 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fadd85-9b64-4eea-adba-d61189fa3b9b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:17.184662 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fadd85-9b64-4eea-adba-d61189fa3b9b None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:17.185987 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 445/891] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:28:17.191364 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9918c7-8dad-493d-80cc-b0703447c836 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:17.191787 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9918c7-8dad-493d-80cc-b0703447c836 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:17.191895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9918c7-8dad-493d-80cc-b0703447c836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:17.210337 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-6e9918c7-8dad-493d-80cc-b0703447c836 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:28:17.212125 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-6e9918c7-8dad-493d-80cc-b0703447c836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:28:17.236453 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:17.276967 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 447/892] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:28:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:28:17.548998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fadd85-9b64-4eea-adba-d61189fa3b9b None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:17.640822 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:17.651372 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fadd85-9b64-4eea-adba-d61189fa3b9b None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:17.684555 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68f68b-29d3-49ca-b4e5-13598c63214e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:17.685166 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68f68b-29d3-49ca-b4e5-13598c63214e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:17.685438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68f68b-29d3-49ca-b4e5-13598c63214e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:17.685706 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68f68b-29d3-49ca-b4e5-13598c63214e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:17.695628 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68f68b-29d3-49ca-b4e5-13598c63214e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:28:17.700747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-ee68f68b-29d3-49ca-b4e5-13598c63214e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:28:17.700747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:28:17.700747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:28:17.700747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:28:17.700747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:28:17.700747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:28:17.700747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:28:17.700747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:28:17.700747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:28:17.700747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:28:17.700747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:28:17.700747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:28:17.700747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:28:17.700747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:28:17.700747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:28:17.700747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:28:17.701899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:28:17.701899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:28:17.701899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:28:17.701899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:28:17.701899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:28:17.701899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:28:17.701899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:28:17.701899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:28:17.701899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:28:17.701899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:28:17.701899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:28:17.701899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:28:17.701899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:28:17.701899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:28:17.701899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:28:17.701899 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:28:17.702811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:28:17.702811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:28:17.702811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:28:17.702811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:28:17.702811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:28:17.702811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:28:17.702811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:28:17.702811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:28:17.702811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:28:17.702811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:28:17.702811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:28:17.702811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:28:17.702811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 446/893] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:28:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:28:17.707864 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9918c7-8dad-493d-80cc-b0703447c836 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:17.801918 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9918c7-8dad-493d-80cc-b0703447c836 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:17.834208 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff00a8-139b-463b-ad72-bd7a25ccb53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:17.834785 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff00a8-139b-463b-ad72-bd7a25ccb53b None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:17.835056 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff00a8-139b-463b-ad72-bd7a25ccb53b None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:17.835348 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff00a8-139b-463b-ad72-bd7a25ccb53b None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:17.835987 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff00a8-139b-463b-ad72-bd7a25ccb53b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:28:17.836322 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff00a8-139b-463b-ad72-bd7a25ccb53b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:17.837661 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff00a8-139b-463b-ad72-bd7a25ccb53b None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:17.845223 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 448/894] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:28:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:28:17.850879 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68f68b-29d3-49ca-b4e5-13598c63214e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:17.947164 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68f68b-29d3-49ca-b4e5-13598c63214e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:17.979211 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ca3dc-6f51-4c9c-9cce-8e3c3968f77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:17.979812 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ca3dc-6f51-4c9c-9cce-8e3c3968f77f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:17.980087 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ca3dc-6f51-4c9c-9cce-8e3c3968f77f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:17.980351 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ca3dc-6f51-4c9c-9cce-8e3c3968f77f None admin] PATH_INFO: `/v3/users/07c12fb724954a53baf9a1f008058328` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:17.993319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ca3dc-6f51-4c9c-9cce-8e3c3968f77f None admin] RBAC: Authorizing `identity:get_user(user_id=07c12fb724954a53baf9a1f008058328)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:17.994790 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ca3dc-6f51-4c9c-9cce-8e3c3968f77f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:17.996352 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 447/895] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:28:17 2022] GET /identity/v3/users/07c12fb724954a53baf9a1f008058328 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:28:18.001961 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff00a8-139b-463b-ad72-bd7a25ccb53b None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:18.027360 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:18.098432 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff00a8-139b-463b-ad72-bd7a25ccb53b None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:18.130129 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a976b73-1979-4caf-8005-4c58812ea98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:18.130734 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a976b73-1979-4caf-8005-4c58812ea98f None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:18.131003 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a976b73-1979-4caf-8005-4c58812ea98f None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:18.131362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a976b73-1979-4caf-8005-4c58812ea98f None admin] PATH_INFO: `/v3/projects/a3a4375fac32481e9eb1d00c12a29b7b` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:18.132442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a976b73-1979-4caf-8005-4c58812ea98f None admin] RBAC: Authorizing `identity:get_project(project_id=a3a4375fac32481e9eb1d00c12a29b7b)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:18.134067 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a976b73-1979-4caf-8005-4c58812ea98f None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:18.135661 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 449/896] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:28:18 2022] GET /identity/v3/projects/a3a4375fac32481e9eb1d00c12a29b7b => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:28:18.141449 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ca3dc-6f51-4c9c-9cce-8e3c3968f77f None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:18.152438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:18.237505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ca3dc-6f51-4c9c-9cce-8e3c3968f77f None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:18.269390 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56107bcb-1807-4493-b7f4-ab9b676a5151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:18.270028 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56107bcb-1807-4493-b7f4-ab9b676a5151 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:18.270303 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56107bcb-1807-4493-b7f4-ab9b676a5151 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:18.270572 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56107bcb-1807-4493-b7f4-ab9b676a5151 None admin] PATH_INFO: `/v3/projects/a3a4375fac32481e9eb1d00c12a29b7b/users/07c12fb724954a53baf9a1f008058328/roles/4b1f48ab911e4fadbc70164ae43d508b` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:18.272454 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56107bcb-1807-4493-b7f4-ab9b676a5151 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3a4375fac32481e9eb1d00c12a29b7b, user_id=07c12fb724954a53baf9a1f008058328, role_id=4b1f48ab911e4fadbc70164ae43d508b)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:18.274368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56107bcb-1807-4493-b7f4-ab9b676a5151 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:18.286390 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 448/897] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:28:18 2022] PUT /identity/v3/projects/a3a4375fac32481e9eb1d00c12a29b7b/users/07c12fb724954a53baf9a1f008058328/roles/4b1f48ab911e4fadbc70164ae43d508b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:28:20.401251 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62cdcc-5ec9-4cf3-8b79-67e89f7472d0 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:20.401783 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62cdcc-5ec9-4cf3-8b79-67e89f7472d0 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:20.401783 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62cdcc-5ec9-4cf3-8b79-67e89f7472d0 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:20.403108 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 450/898] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:28:20.408486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4799a-4b55-4d99-9584-a250c3075f86 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:20.408886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4799a-4b55-4d99-9584-a250c3075f86 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:20.409016 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4799a-4b55-4d99-9584-a250c3075f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:20.427104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d9c4799a-4b55-4d99-9584-a250c3075f86 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:28:20.428292 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d9c4799a-4b55-4d99-9584-a250c3075f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:28:20.486178 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:20.492130 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 449/899] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:28:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:28:20.759078 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62cdcc-5ec9-4cf3-8b79-67e89f7472d0 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:20.847923 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:20.858112 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62cdcc-5ec9-4cf3-8b79-67e89f7472d0 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:20.890694 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a38437-45bc-4bd4-afa9-718a30dfbbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:20.891455 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a38437-45bc-4bd4-afa9-718a30dfbbad None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:20.891732 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a38437-45bc-4bd4-afa9-718a30dfbbad None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:20.891996 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a38437-45bc-4bd4-afa9-718a30dfbbad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:20.902645 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a38437-45bc-4bd4-afa9-718a30dfbbad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:28:20.907689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-f4a38437-45bc-4bd4-afa9-718a30dfbbad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:28:20.907689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:28:20.907689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:28:20.907689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:28:20.907689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:28:20.907689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:28:20.907689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:28:20.907689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:28:20.907689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:28:20.907689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:28:20.907689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:28:20.907689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:28:20.907689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:28:20.907689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:28:20.907689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:28:20.907689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:28:20.908746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:28:20.908746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:28:20.908746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:28:20.908746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:28:20.908746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:28:20.908746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:28:20.908746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:28:20.908746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:28:20.908746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:28:20.908746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:28:20.908746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:28:20.908746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:28:20.908746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:28:20.908746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:28:20.908746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:28:20.908746 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:28:20.909693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:28:20.909693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:28:20.909693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:28:20.909693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:28:20.909693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:28:20.909693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:28:20.909693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:28:20.909693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:28:20.909693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:28:20.909693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:28:20.909693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:28:20.909693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:28:20.909693 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 451/900] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:28:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:28:20.914894 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4799a-4b55-4d99-9584-a250c3075f86 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:21.009796 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4799a-4b55-4d99-9584-a250c3075f86 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:21.041830 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f8d69-1f20-4395-a780-ef3ea54854c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:21.042407 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f8d69-1f20-4395-a780-ef3ea54854c2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:21.042676 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f8d69-1f20-4395-a780-ef3ea54854c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:21.042941 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f8d69-1f20-4395-a780-ef3ea54854c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:21.043605 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f8d69-1f20-4395-a780-ef3ea54854c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:28:21.043897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f8d69-1f20-4395-a780-ef3ea54854c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:21.045235 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f8d69-1f20-4395-a780-ef3ea54854c2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:21.052851 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 450/901] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:28:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:28:21.059104 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a38437-45bc-4bd4-afa9-718a30dfbbad None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:21.156022 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a38437-45bc-4bd4-afa9-718a30dfbbad None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:21.187819 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a855d1-e14b-424a-a5a0-a9328a5bb0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:21.188428 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a855d1-e14b-424a-a5a0-a9328a5bb0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:21.188703 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a855d1-e14b-424a-a5a0-a9328a5bb0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:21.188968 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a855d1-e14b-424a-a5a0-a9328a5bb0e9 None admin] PATH_INFO: `/v3/users/07c12fb724954a53baf9a1f008058328` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:21.190052 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a855d1-e14b-424a-a5a0-a9328a5bb0e9 None admin] RBAC: Authorizing `identity:get_user(user_id=07c12fb724954a53baf9a1f008058328)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:21.191491 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a855d1-e14b-424a-a5a0-a9328a5bb0e9 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:21.193060 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 452/902] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Oct 12 14:28:21 2022] GET /identity/v3/users/07c12fb724954a53baf9a1f008058328 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:28:21.199481 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f8d69-1f20-4395-a780-ef3ea54854c2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:21.255387 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:21.294862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f8d69-1f20-4395-a780-ef3ea54854c2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:21.326915 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e90a4-c04f-48fc-8109-051817123862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:21.327533 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e90a4-c04f-48fc-8109-051817123862 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:21.327806 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e90a4-c04f-48fc-8109-051817123862 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:21.328073 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e90a4-c04f-48fc-8109-051817123862 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:21.391687 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e90a4-c04f-48fc-8109-051817123862 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:21.393108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e90a4-c04f-48fc-8109-051817123862 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:21.447491 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:21.487111 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 451/903] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:28:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 14:28:21.493396 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a855d1-e14b-424a-a5a0-a9328a5bb0e9 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:21.515079 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:21.591929 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a855d1-e14b-424a-a5a0-a9328a5bb0e9 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:21.624004 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5a60c-6c26-4d65-ade2-64bbaf59d033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:21.624613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5a60c-6c26-4d65-ade2-64bbaf59d033 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:21.624885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5a60c-6c26-4d65-ade2-64bbaf59d033 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:21.625149 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5a60c-6c26-4d65-ade2-64bbaf59d033 None admin] PATH_INFO: `/v3/projects/a3a4375fac32481e9eb1d00c12a29b7b` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:21.626238 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5a60c-6c26-4d65-ade2-64bbaf59d033 None admin] RBAC: Authorizing `identity:get_project(project_id=a3a4375fac32481e9eb1d00c12a29b7b)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:21.627682 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5a60c-6c26-4d65-ade2-64bbaf59d033 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:21.629305 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 453/904] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Oct 12 14:28:21 2022] GET /identity/v3/projects/a3a4375fac32481e9eb1d00c12a29b7b => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:28:21.635947 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e90a4-c04f-48fc-8109-051817123862 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:28:21.729625 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e90a4-c04f-48fc-8109-051817123862 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:28:21.760970 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51188b-22a3-451e-b6d7-677c8bb50181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:28:21.761549 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51188b-22a3-451e-b6d7-677c8bb50181 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:28:21.761819 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51188b-22a3-451e-b6d7-677c8bb50181 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:28:21.762082 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51188b-22a3-451e-b6d7-677c8bb50181 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:28:21.762784 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51188b-22a3-451e-b6d7-677c8bb50181 None admin] RBAC: Adding query filter params (role.id=4b1f48ab911e4fadbc70164ae43d508b, scope.project.id=a3a4375fac32481e9eb1d00c12a29b7b, user.id=07c12fb724954a53baf9a1f008058328) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:28:21.763081 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51188b-22a3-451e-b6d7-677c8bb50181 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:28:21.764498 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51188b-22a3-451e-b6d7-677c8bb50181 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:28:21.769925 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 452/905] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:28:21 2022] GET /identity/v3/role_assignments?user.id=07c12fb724954a53baf9a1f008058328&scope.project.id=a3a4375fac32481e9eb1d00c12a29b7b&role.id=4b1f48ab911e4fadbc70164ae43d508b => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:28:31.620080 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=56 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:28:31.767501 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=88 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:17.231856 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e036f0-9460-4281-8d5b-46fee02fb790 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:17.232481 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e036f0-9460-4281-8d5b-46fee02fb790 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:17.232481 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e036f0-9460-4281-8d5b-46fee02fb790 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:17.233759 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 454/906] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:29:17.243188 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51125419-5d98-4018-bdc0-44b43b8de803 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:17.243188 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51125419-5d98-4018-bdc0-44b43b8de803 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:17.243188 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51125419-5d98-4018-bdc0-44b43b8de803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:17.263705 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-51125419-5d98-4018-bdc0-44b43b8de803 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:29:17.265143 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-51125419-5d98-4018-bdc0-44b43b8de803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:29:17.355193 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 453/907] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:29:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 14:29:17.638086 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e036f0-9460-4281-8d5b-46fee02fb790 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:17.778370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e036f0-9460-4281-8d5b-46fee02fb790 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:17.816619 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ecb14f-c43e-4d64-809d-9c24db2a8e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:17.817421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecb14f-c43e-4d64-809d-9c24db2a8e90 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:17.817630 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecb14f-c43e-4d64-809d-9c24db2a8e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:17.817900 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecb14f-c43e-4d64-809d-9c24db2a8e90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:17.830828 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ecb14f-c43e-4d64-809d-9c24db2a8e90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:17.832531 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ecb14f-c43e-4d64-809d-9c24db2a8e90 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:17.852055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-f5ecb14f-c43e-4d64-809d-9c24db2a8e90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:17.852055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:17.852055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:17.852055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:17.852055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:17.852055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:17.852055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:17.852055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:17.852055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:17.852055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:17.852055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:17.852055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:17.852055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:29:17.852055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:29:17.852055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:29:17.852055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:17.853397 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:29:17.854329 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:29:17.854329 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:17.854329 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:29:17.861847 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 455/908] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:17.862024 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51125419-5d98-4018-bdc0-44b43b8de803 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:17.990440 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51125419-5d98-4018-bdc0-44b43b8de803 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:18.036614 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:18.043046 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01284f11-8730-4d5c-9eda-9e9ea5bc7707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:18.043704 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01284f11-8730-4d5c-9eda-9e9ea5bc7707 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:18.043989 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01284f11-8730-4d5c-9eda-9e9ea5bc7707 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:18.044307 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01284f11-8730-4d5c-9eda-9e9ea5bc7707 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:18.044989 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01284f11-8730-4d5c-9eda-9e9ea5bc7707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:18.045354 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01284f11-8730-4d5c-9eda-9e9ea5bc7707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:18.046761 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01284f11-8730-4d5c-9eda-9e9ea5bc7707 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:18.067496 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 454/909] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:18.068898 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ecb14f-c43e-4d64-809d-9c24db2a8e90 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:18.134329 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:18.198734 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ecb14f-c43e-4d64-809d-9c24db2a8e90 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:18.245737 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67bf1e-35b0-416b-9dc8-41e85612f95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:18.246511 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67bf1e-35b0-416b-9dc8-41e85612f95e None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:18.246792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67bf1e-35b0-416b-9dc8-41e85612f95e None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:18.247112 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67bf1e-35b0-416b-9dc8-41e85612f95e None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:18.247906 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67bf1e-35b0-416b-9dc8-41e85612f95e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:18.249422 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67bf1e-35b0-416b-9dc8-41e85612f95e None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:18.276943 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 456/910] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Oct 12 14:29:18 2022] POST /identity/v3/users => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 14:29:20.937981 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cf6ed-4f03-45d2-ae51-d0668c9cb923 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:20.938528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cf6ed-4f03-45d2-ae51-d0668c9cb923 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:20.938638 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cf6ed-4f03-45d2-ae51-d0668c9cb923 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:20.940238 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 455/911] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:29:20.946747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17052cc4-48b3-4222-9f72-f9ed5e011fd5 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:20.947401 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17052cc4-48b3-4222-9f72-f9ed5e011fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:20.947401 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17052cc4-48b3-4222-9f72-f9ed5e011fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:20.969125 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-17052cc4-48b3-4222-9f72-f9ed5e011fd5 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:29:20.970506 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-17052cc4-48b3-4222-9f72-f9ed5e011fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:29:21.063187 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 457/912] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:29:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:29:21.374656 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cf6ed-4f03-45d2-ae51-d0668c9cb923 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:21.494217 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cf6ed-4f03-45d2-ae51-d0668c9cb923 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:21.532120 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a12a7-4f9c-4ecc-a4f1-4a23c01a1c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:21.532758 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a12a7-4f9c-4ecc-a4f1-4a23c01a1c65 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:21.533030 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a12a7-4f9c-4ecc-a4f1-4a23c01a1c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:21.533291 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a12a7-4f9c-4ecc-a4f1-4a23c01a1c65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:21.545469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a12a7-4f9c-4ecc-a4f1-4a23c01a1c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:29:21.550252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-2e9a12a7-4f9c-4ecc-a4f1-4a23c01a1c65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:29:21.550252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:21.550252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:21.550252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:21.550252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:21.550252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:21.550252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:21.550252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:21.550252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:21.550252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:21.550252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:21.550252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:21.550252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:29:21.550252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:29:21.550252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:29:21.550252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:29:21.551309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:29:21.551309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:29:21.551309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:29:21.551309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:29:21.551309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:21.551309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:21.551309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:21.551309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:21.551309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:21.551309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:21.551309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:21.551309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:21.551309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:21.551309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:21.551309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:21.551309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:21.552211 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:21.552211 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:21.552211 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:21.552211 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:21.552211 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:29:21.552211 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:29:21.552211 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:29:21.552211 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:29:21.552211 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:29:21.552211 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:29:21.552211 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:29:21.552211 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:21.552211 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 456/913] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Oct 12 14:29:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:21.560134 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17052cc4-48b3-4222-9f72-f9ed5e011fd5 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:21.605841 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:21.678289 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17052cc4-48b3-4222-9f72-f9ed5e011fd5 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:21.717504 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e7344-f4b9-4277-9916-c1c9b71481fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:21.718110 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e7344-f4b9-4277-9916-c1c9b71481fe None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:21.718384 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e7344-f4b9-4277-9916-c1c9b71481fe None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:21.718650 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e7344-f4b9-4277-9916-c1c9b71481fe None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:21.719336 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e7344-f4b9-4277-9916-c1c9b71481fe None admin] RBAC: Adding query filter params (name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:21.719636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e7344-f4b9-4277-9916-c1c9b71481fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:21.721022 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e7344-f4b9-4277-9916-c1c9b71481fe None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:21.734061 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 458/914] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Oct 12 14:29:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:21.735300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a12a7-4f9c-4ecc-a4f1-4a23c01a1c65 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:21.800688 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:21.874783 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a12a7-4f9c-4ecc-a4f1-4a23c01a1c65 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:21.913047 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee7536-c78c-4b48-b79b-d3dfdda1b086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:21.913712 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee7536-c78c-4b48-b79b-d3dfdda1b086 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:21.913985 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee7536-c78c-4b48-b79b-d3dfdda1b086 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:21.914259 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee7536-c78c-4b48-b79b-d3dfdda1b086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:21.927460 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee7536-c78c-4b48-b79b-d3dfdda1b086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:21.929933 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee7536-c78c-4b48-b79b-d3dfdda1b086 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:21.946720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-ccee7536-c78c-4b48-b79b-d3dfdda1b086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:21.946720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:21.946720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:21.946720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:21.946720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:21.946720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:21.946720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:21.946720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:21.946720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:21.946720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:21.946720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:21.946720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:21.946720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:29:21.946720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:29:21.946720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:29:21.946720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:21.952963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:29:21.953832 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:29:21.953832 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:21.953832 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:21.953832 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 457/915] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:29:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:21.954081 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e7344-f4b9-4277-9916-c1c9b71481fe None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:22.067287 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:22.072198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e7344-f4b9-4277-9916-c1c9b71481fe None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:22.113384 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f03f75-6b4b-438b-84ee-ec252bbb4a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:22.113982 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f03f75-6b4b-438b-84ee-ec252bbb4a21 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:22.114252 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f03f75-6b4b-438b-84ee-ec252bbb4a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:22.114515 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f03f75-6b4b-438b-84ee-ec252bbb4a21 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:22.115163 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f03f75-6b4b-438b-84ee-ec252bbb4a21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:22.115463 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f03f75-6b4b-438b-84ee-ec252bbb4a21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:22.117152 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f03f75-6b4b-438b-84ee-ec252bbb4a21 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:22.129227 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 459/916] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:22.136129 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee7536-c78c-4b48-b79b-d3dfdda1b086 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:22.368745 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee7536-c78c-4b48-b79b-d3dfdda1b086 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:22.370432 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:22.411752 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476cf42-864b-48b0-a6f0-35756ce3e6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:22.412382 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476cf42-864b-48b0-a6f0-35756ce3e6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:22.412656 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476cf42-864b-48b0-a6f0-35756ce3e6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:22.412916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476cf42-864b-48b0-a6f0-35756ce3e6c1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:22.423369 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3476cf42-864b-48b0-a6f0-35756ce3e6c1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:22.424876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3476cf42-864b-48b0-a6f0-35756ce3e6c1 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:22.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-3476cf42-864b-48b0-a6f0-35756ce3e6c1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 12 14:29:22.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:22.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:22.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:22.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:22.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:22.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:22.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:22.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:22.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:22.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:22.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:22.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:29:22.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:29:22.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:29:22.436750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:22.437760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:22.438771 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:22.438771 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:22.438771 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:29:22.438771 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:29:22.438771 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:29:22.438771 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:29:22.438771 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:29:22.438771 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:29:22.438771 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 12 14:29:22.438771 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:22.438771 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 458/917] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:29:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:22.443731 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f03f75-6b4b-438b-84ee-ec252bbb4a21 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:22.553556 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f03f75-6b4b-438b-84ee-ec252bbb4a21 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:22.599347 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b72bf8-04ae-4a61-b2e1-935e0f3f2cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:22.599979 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b72bf8-04ae-4a61-b2e1-935e0f3f2cfe None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:22.600263 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b72bf8-04ae-4a61-b2e1-935e0f3f2cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:22.600601 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b72bf8-04ae-4a61-b2e1-935e0f3f2cfe None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:22.602137 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b72bf8-04ae-4a61-b2e1-935e0f3f2cfe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:22.602408 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b72bf8-04ae-4a61-b2e1-935e0f3f2cfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:22.604024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b72bf8-04ae-4a61-b2e1-935e0f3f2cfe None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:22.623444 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 460/918] 10.222.0.16 () {56 vars in 1143 bytes} [Wed Oct 12 14:29:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:22.629776 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476cf42-864b-48b0-a6f0-35756ce3e6c1 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:22.746910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476cf42-864b-48b0-a6f0-35756ce3e6c1 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:22.792266 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ec54c-5fb1-458d-9dcb-fd97fbc9ee4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:22.793159 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ec54c-5fb1-458d-9dcb-fd97fbc9ee4e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:22.793539 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ec54c-5fb1-458d-9dcb-fd97fbc9ee4e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:22.793830 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ec54c-5fb1-458d-9dcb-fd97fbc9ee4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:22.807028 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ec54c-5fb1-458d-9dcb-fd97fbc9ee4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:22.808743 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ec54c-5fb1-458d-9dcb-fd97fbc9ee4e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:22.827074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-c52ec54c-5fb1-458d-9dcb-fd97fbc9ee4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:22.827074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:22.827074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:22.827074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:22.827074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:22.827074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:22.827074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:22.827074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:22.827074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:22.827074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:22.827074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:22.827074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:22.827074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:29:22.827074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:29:22.827074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:29:22.827074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:22.828361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:29:22.829427 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:29:22.829427 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:22.829427 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:22.829427 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 459/919] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:29:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:22.836947 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b72bf8-04ae-4a61-b2e1-935e0f3f2cfe None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:22.858488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:22.966105 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b72bf8-04ae-4a61-b2e1-935e0f3f2cfe None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:23.010055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f138ab-691f-4b6b-ba2d-a54532e315fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:23.010677 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f138ab-691f-4b6b-ba2d-a54532e315fa None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:23.010956 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f138ab-691f-4b6b-ba2d-a54532e315fa None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:23.011234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f138ab-691f-4b6b-ba2d-a54532e315fa None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:23.011884 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f138ab-691f-4b6b-ba2d-a54532e315fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:23.012177 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f138ab-691f-4b6b-ba2d-a54532e315fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:23.013587 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f138ab-691f-4b6b-ba2d-a54532e315fa None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:23.028447 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 461/920] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:29:23.034946 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ec54c-5fb1-458d-9dcb-fd97fbc9ee4e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:23.058538 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:23.153185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ec54c-5fb1-458d-9dcb-fd97fbc9ee4e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:23.193756 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc85203-7efc-477a-9622-b27804250b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:23.195107 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc85203-7efc-477a-9622-b27804250b2a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:23.195498 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc85203-7efc-477a-9622-b27804250b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:23.195783 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc85203-7efc-477a-9622-b27804250b2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:23.202736 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc85203-7efc-477a-9622-b27804250b2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:23.204259 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc85203-7efc-477a-9622-b27804250b2a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:23.218420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-3bc85203-7efc-477a-9622-b27804250b2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:29:23.218420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:23.218420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:23.218420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:23.218420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:23.218420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:23.218420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:23.218420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:23.218420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:23.218420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:23.218420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:23.218420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:23.218420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:29:23.218420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:29:23.218420 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:23.219657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:23.220886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:29:23.220886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:29:23.220886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:29:23.220886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:29:23.220886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:29:23.220886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:29:23.220886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:29:23.220886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:23.220886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 460/921] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:29:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:23.225907 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f138ab-691f-4b6b-ba2d-a54532e315fa None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:23.315547 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:23.343529 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f138ab-691f-4b6b-ba2d-a54532e315fa None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:23.383211 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ca437-d067-4757-9be3-1e58d7dd9120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:23.383829 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ca437-d067-4757-9be3-1e58d7dd9120 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:23.384104 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ca437-d067-4757-9be3-1e58d7dd9120 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:23.384376 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ca437-d067-4757-9be3-1e58d7dd9120 None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:23.385052 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ca437-d067-4757-9be3-1e58d7dd9120 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:23.385350 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ca437-d067-4757-9be3-1e58d7dd9120 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:23.386757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ca437-d067-4757-9be3-1e58d7dd9120 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:23.399391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 462/922] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:29:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:23.405798 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc85203-7efc-477a-9622-b27804250b2a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:23.492229 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:23.517474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc85203-7efc-477a-9622-b27804250b2a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:23.552964 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94520bd5-5cf3-4a03-ad7e-0856820ee671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:23.553609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94520bd5-5cf3-4a03-ad7e-0856820ee671 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:23.553941 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94520bd5-5cf3-4a03-ad7e-0856820ee671 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:23.554264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94520bd5-5cf3-4a03-ad7e-0856820ee671 None admin] PATH_INFO: `/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/93a6d6ccbf4940a9b292be17a94bfac8/roles/745dcd2d05e447538a1aaff54b38193c` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:23.570708 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94520bd5-5cf3-4a03-ad7e-0856820ee671 None admin] RBAC: Authorizing `identity:create_grant(project_id=720b11237d2e48128c171be45fcd0ad0, user_id=93a6d6ccbf4940a9b292be17a94bfac8, role_id=745dcd2d05e447538a1aaff54b38193c)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:23.573436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94520bd5-5cf3-4a03-ad7e-0856820ee671 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:23.591347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 461/923] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:29:23 2022] PUT /identity/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/93a6d6ccbf4940a9b292be17a94bfac8/roles/745dcd2d05e447538a1aaff54b38193c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 14:29:25.865434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8e460-5195-4d71-8c3c-0615a5dbb1d3 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:25.865953 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8e460-5195-4d71-8c3c-0615a5dbb1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:25.865953 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8e460-5195-4d71-8c3c-0615a5dbb1d3 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:25.868409 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 463/924] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:29:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:29:25.877044 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ba437-4a38-427b-8d39-8355bcf472e1 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:25.877531 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ba437-4a38-427b-8d39-8355bcf472e1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:25.877635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ba437-4a38-427b-8d39-8355bcf472e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:25.895846 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-f46ba437-4a38-427b-8d39-8355bcf472e1 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:29:25.897021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f46ba437-4a38-427b-8d39-8355bcf472e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:29:25.981884 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 462/925] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:29:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:29:26.267981 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8e460-5195-4d71-8c3c-0615a5dbb1d3 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:26.374554 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8e460-5195-4d71-8c3c-0615a5dbb1d3 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:26.410851 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ed1dd-f9e0-4ee4-a6e8-c7396a6597c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:26.411535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ed1dd-f9e0-4ee4-a6e8-c7396a6597c2 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:26.411868 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ed1dd-f9e0-4ee4-a6e8-c7396a6597c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:26.412080 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ed1dd-f9e0-4ee4-a6e8-c7396a6597c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:26.420192 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:26.424990 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31ed1dd-f9e0-4ee4-a6e8-c7396a6597c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:29:26.432198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-f31ed1dd-f9e0-4ee4-a6e8-c7396a6597c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:29:26.432198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:26.432198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:26.432198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:26.432198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:26.432198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:26.432198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:26.432198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:26.432198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:26.432198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:26.432198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:26.432198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:26.432198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:29:26.432198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:29:26.432198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:29:26.432198 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:29:26.433340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:29:26.433340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:29:26.433340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:29:26.433340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:29:26.433340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:26.433340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:26.433340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:26.433340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:26.433340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:26.433340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:26.433340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:26.433340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:26.433340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:26.433340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:26.433340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:26.433340 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:26.434391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:26.434391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:26.434391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:26.434391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:26.434391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:29:26.434391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:29:26.434391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:29:26.434391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:29:26.434391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:29:26.434391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:29:26.434391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 14:29:26.434391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:29:26.434391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 464/926] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Oct 12 14:29:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:26.439715 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ba437-4a38-427b-8d39-8355bcf472e1 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:26.498722 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:26.551755 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ba437-4a38-427b-8d39-8355bcf472e1 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:26.590273 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb338e-5e2c-4e3b-844e-fac3634bf605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:26.590891 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb338e-5e2c-4e3b-844e-fac3634bf605 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:26.591171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb338e-5e2c-4e3b-844e-fac3634bf605 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:26.591435 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb338e-5e2c-4e3b-844e-fac3634bf605 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:26.592096 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb338e-5e2c-4e3b-844e-fac3634bf605 None admin] RBAC: Adding query filter params (name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:26.592413 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb338e-5e2c-4e3b-844e-fac3634bf605 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:26.594058 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb338e-5e2c-4e3b-844e-fac3634bf605 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:26.604018 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 463/927] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Oct 12 14:29:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:26.611189 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ed1dd-f9e0-4ee4-a6e8-c7396a6597c2 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:26.724614 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ed1dd-f9e0-4ee4-a6e8-c7396a6597c2 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:26.761744 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4f176-16c4-4e08-b9b6-d8c85f3049ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:26.762369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4f176-16c4-4e08-b9b6-d8c85f3049ce None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:26.762641 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4f176-16c4-4e08-b9b6-d8c85f3049ce None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:26.762906 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4f176-16c4-4e08-b9b6-d8c85f3049ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:26.773716 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4f176-16c4-4e08-b9b6-d8c85f3049ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:26.776482 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4f176-16c4-4e08-b9b6-d8c85f3049ce None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:26.792966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-a5b4f176-16c4-4e08-b9b6-d8c85f3049ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:26.792966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:26.792966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:26.792966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:26.792966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:26.792966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:26.792966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:26.792966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:26.792966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:26.792966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:26.792966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:26.792966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:26.792966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:29:26.792966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:29:26.792966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:29:26.792966 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:26.794084 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:29:26.795195 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:29:26.795195 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:26.795195 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:29:26.795195 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 465/928] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:29:26.800422 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb338e-5e2c-4e3b-844e-fac3634bf605 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:26.904760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb338e-5e2c-4e3b-844e-fac3634bf605 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:26.926576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:26.946347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c13f9-23eb-43cc-ace6-0d730675c68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:26.946956 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c13f9-23eb-43cc-ace6-0d730675c68d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:26.947347 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c13f9-23eb-43cc-ace6-0d730675c68d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:26.947529 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c13f9-23eb-43cc-ace6-0d730675c68d None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:26.948165 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c13f9-23eb-43cc-ace6-0d730675c68d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:26.948457 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c13f9-23eb-43cc-ace6-0d730675c68d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:26.950007 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c13f9-23eb-43cc-ace6-0d730675c68d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:26.962047 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 464/929] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:26.968861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4f176-16c4-4e08-b9b6-d8c85f3049ce None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:27.029055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:27.090311 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4f176-16c4-4e08-b9b6-d8c85f3049ce None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:27.127724 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a3e05-4b0c-47a4-9f3f-076b9fc077a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:27.128338 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a3e05-4b0c-47a4-9f3f-076b9fc077a1 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:27.128615 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a3e05-4b0c-47a4-9f3f-076b9fc077a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:27.128910 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a3e05-4b0c-47a4-9f3f-076b9fc077a1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:27.137044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a3e05-4b0c-47a4-9f3f-076b9fc077a1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:27.138468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a3e05-4b0c-47a4-9f3f-076b9fc077a1 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:27.149788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-d19a3e05-4b0c-47a4-9f3f-076b9fc077a1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 12 14:29:27.149788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:27.149788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:27.149788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:27.149788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:27.149788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:27.149788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:27.149788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:27.149788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:27.149788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:27.149788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:27.149788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:27.149788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:29:27.149788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:29:27.149788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:29:27.149788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:27.150876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:27.151836 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:27.151836 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:27.151836 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:29:27.151836 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:29:27.151836 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:29:27.151836 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:29:27.151836 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:29:27.151836 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:29:27.151836 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 12 14:29:27.151836 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:29:27.156095 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 466/930] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:29:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:27.157073 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c13f9-23eb-43cc-ace6-0d730675c68d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:27.277787 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c13f9-23eb-43cc-ace6-0d730675c68d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:27.311976 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64029c38-091a-4223-9f7c-c96f16966b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:27.312576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64029c38-091a-4223-9f7c-c96f16966b22 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:27.312845 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64029c38-091a-4223-9f7c-c96f16966b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:27.313108 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64029c38-091a-4223-9f7c-c96f16966b22 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:27.313823 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64029c38-091a-4223-9f7c-c96f16966b22 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:27.314115 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64029c38-091a-4223-9f7c-c96f16966b22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:27.315553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64029c38-091a-4223-9f7c-c96f16966b22 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:27.340927 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 465/931] 10.222.0.16 () {56 vars in 1143 bytes} [Wed Oct 12 14:29:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:29:27.342141 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a3e05-4b0c-47a4-9f3f-076b9fc077a1 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:27.446058 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a3e05-4b0c-47a4-9f3f-076b9fc077a1 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:27.455980 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:27.481669 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51670bf-6330-4aba-adcd-e0d4f91de04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:27.482325 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51670bf-6330-4aba-adcd-e0d4f91de04b None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:27.482581 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51670bf-6330-4aba-adcd-e0d4f91de04b None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:27.482914 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51670bf-6330-4aba-adcd-e0d4f91de04b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:27.555943 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51670bf-6330-4aba-adcd-e0d4f91de04b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:27.557589 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51670bf-6330-4aba-adcd-e0d4f91de04b None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:27.678357 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 467/932] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:29:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 14:29:27.685987 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64029c38-091a-4223-9f7c-c96f16966b22 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:27.722107 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:27.804881 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64029c38-091a-4223-9f7c-c96f16966b22 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:27.842350 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bea35-09b4-45a6-9fd2-a86f28a01536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:27.843021 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bea35-09b4-45a6-9fd2-a86f28a01536 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:27.843327 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bea35-09b4-45a6-9fd2-a86f28a01536 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:27.843595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bea35-09b4-45a6-9fd2-a86f28a01536 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:27.854904 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9bea35-09b4-45a6-9fd2-a86f28a01536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:27.856377 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9bea35-09b4-45a6-9fd2-a86f28a01536 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:27.871574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-bf9bea35-09b4-45a6-9fd2-a86f28a01536 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:27.871574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:27.871574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:27.871574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:27.871574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:27.871574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:27.871574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:27.871574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:27.871574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:27.871574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:27.871574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:27.871574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:27.871574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:29:27.871574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:29:27.871574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:29:27.871574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:27.872733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:29:27.873661 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:29:27.873661 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:27.873661 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:27.873661 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 466/933] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 14:29:27.879407 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51670bf-6330-4aba-adcd-e0d4f91de04b None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:27.890805 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:27.988499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51670bf-6330-4aba-adcd-e0d4f91de04b None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:28.027416 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021e78d-b8cb-4807-95f5-cf9cf6745ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:28.028034 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021e78d-b8cb-4807-95f5-cf9cf6745ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:28.028386 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021e78d-b8cb-4807-95f5-cf9cf6745ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:28.028658 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021e78d-b8cb-4807-95f5-cf9cf6745ed0 None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:28.029303 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021e78d-b8cb-4807-95f5-cf9cf6745ed0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:28.029601 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021e78d-b8cb-4807-95f5-cf9cf6745ed0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:28.031243 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021e78d-b8cb-4807-95f5-cf9cf6745ed0 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:28.044905 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 468/934] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:28.050981 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bea35-09b4-45a6-9fd2-a86f28a01536 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:28.145016 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:28.165482 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bea35-09b4-45a6-9fd2-a86f28a01536 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:28.204365 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac88d44-4b23-4a21-b09e-0c3c2af60f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:28.205088 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac88d44-4b23-4a21-b09e-0c3c2af60f37 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:28.205367 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac88d44-4b23-4a21-b09e-0c3c2af60f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:28.205641 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac88d44-4b23-4a21-b09e-0c3c2af60f37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:28.213154 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac88d44-4b23-4a21-b09e-0c3c2af60f37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:28.214919 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac88d44-4b23-4a21-b09e-0c3c2af60f37 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:28.229811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-eac88d44-4b23-4a21-b09e-0c3c2af60f37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:29:28.229811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:28.229811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:28.229811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:28.229811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:28.229811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:28.229811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:28.229811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:28.229811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:28.229811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:28.229811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:28.229811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:28.229811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:29:28.229811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:29:28.229811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:28.230858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:28.231955 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:29:28.231955 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:29:28.231955 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:29:28.231955 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:29:28.231955 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:29:28.231955 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:29:28.231955 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:29:28.231955 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:28.231955 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 467/935] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:29:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:28.239442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021e78d-b8cb-4807-95f5-cf9cf6745ed0 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:28.313297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:28.346188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021e78d-b8cb-4807-95f5-cf9cf6745ed0 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:28.382839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495264b4-936e-4bc8-a00d-0e12e4887b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:28.383480 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495264b4-936e-4bc8-a00d-0e12e4887b5b None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:28.383820 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495264b4-936e-4bc8-a00d-0e12e4887b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:28.384232 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495264b4-936e-4bc8-a00d-0e12e4887b5b None admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:28.385314 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495264b4-936e-4bc8-a00d-0e12e4887b5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:28.385824 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495264b4-936e-4bc8-a00d-0e12e4887b5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:28.387582 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495264b4-936e-4bc8-a00d-0e12e4887b5b None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:28.402971 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 469/936] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:29:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:29:28.411168 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac88d44-4b23-4a21-b09e-0c3c2af60f37 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:28.518518 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac88d44-4b23-4a21-b09e-0c3c2af60f37 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:28.555988 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951325c-01f1-4696-84ce-ecd3db3a7cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:28.556591 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951325c-01f1-4696-84ce-ecd3db3a7cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:28.556865 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951325c-01f1-4696-84ce-ecd3db3a7cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:28.557132 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951325c-01f1-4696-84ce-ecd3db3a7cf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:28.557853 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951325c-01f1-4696-84ce-ecd3db3a7cf5 None admin] RBAC: Adding query filter params (role.id=745dcd2d05e447538a1aaff54b38193c, scope.project.id=720b11237d2e48128c171be45fcd0ad0, user.id=93a6d6ccbf4940a9b292be17a94bfac8) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:28.558156 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951325c-01f1-4696-84ce-ecd3db3a7cf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:28.559610 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951325c-01f1-4696-84ce-ecd3db3a7cf5 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:28.567110 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 468/937] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:29:28 2022] GET /identity/v3/role_assignments?user.id=93a6d6ccbf4940a9b292be17a94bfac8&scope.project.id=720b11237d2e48128c171be45fcd0ad0&role.id=745dcd2d05e447538a1aaff54b38193c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:30.861297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec391b-6816-4a8e-8967-007a265d71f1 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:30.861871 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec391b-6816-4a8e-8967-007a265d71f1 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:30.861871 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec391b-6816-4a8e-8967-007a265d71f1 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:30.863390 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 470/938] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:29:30.869411 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8f3ca-b9a3-4eee-be28-af9ed0a653b6 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:30.870026 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8f3ca-b9a3-4eee-be28-af9ed0a653b6 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:30.870026 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8f3ca-b9a3-4eee-be28-af9ed0a653b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:30.888517 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3ed8f3ca-b9a3-4eee-be28-af9ed0a653b6 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:29:30.889681 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3ed8f3ca-b9a3-4eee-be28-af9ed0a653b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:29:30.892374 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:30.958296 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 469/939] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:29:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:29:31.264418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec391b-6816-4a8e-8967-007a265d71f1 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:31.376895 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec391b-6816-4a8e-8967-007a265d71f1 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:31.410765 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:31.417780 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c3942-5d39-4270-9b86-321e86f4521e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:31.418545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c3942-5d39-4270-9b86-321e86f4521e None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:31.418912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c3942-5d39-4270-9b86-321e86f4521e None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:31.419259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c3942-5d39-4270-9b86-321e86f4521e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:31.430599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c3942-5d39-4270-9b86-321e86f4521e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:29:31.435625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-5f1c3942-5d39-4270-9b86-321e86f4521e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:29:31.435625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:31.435625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:31.435625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:31.435625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:31.435625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:31.435625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:31.435625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:31.435625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:31.435625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:31.435625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:31.435625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:31.435625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:29:31.435625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:29:31.435625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:29:31.435625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:29:31.436802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:29:31.436802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:29:31.436802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:29:31.436802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:29:31.436802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:31.436802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:31.436802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:31.436802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:31.436802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:31.436802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:31.436802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:31.436802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:31.436802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:31.436802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:31.436802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:31.436802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:31.437733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:31.437733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:31.437733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:31.437733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:31.437733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:29:31.437733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:29:31.437733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:29:31.437733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:29:31.437733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:29:31.437733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:29:31.437733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:29:31.437733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:29:31.437733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 471/940] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:29:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:31.442976 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8f3ca-b9a3-4eee-be28-af9ed0a653b6 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:31.552070 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8f3ca-b9a3-4eee-be28-af9ed0a653b6 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:31.591690 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879f7d0-533e-4c12-a93b-425d342c25ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:31.592310 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879f7d0-533e-4c12-a93b-425d342c25ee None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:31.592591 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879f7d0-533e-4c12-a93b-425d342c25ee None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:31.592897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879f7d0-533e-4c12-a93b-425d342c25ee None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:31.593583 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5879f7d0-533e-4c12-a93b-425d342c25ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:31.593884 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5879f7d0-533e-4c12-a93b-425d342c25ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:31.595385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5879f7d0-533e-4c12-a93b-425d342c25ee None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:31.604503 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 470/941] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:29:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:31.610471 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c3942-5d39-4270-9b86-321e86f4521e None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:31.720985 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c3942-5d39-4270-9b86-321e86f4521e None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:31.761576 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b916db-7401-438b-8f4b-e964e21c4a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:31.762461 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b916db-7401-438b-8f4b-e964e21c4a35 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:31.762741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b916db-7401-438b-8f4b-e964e21c4a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:31.763083 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b916db-7401-438b-8f4b-e964e21c4a35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:31.774963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b916db-7401-438b-8f4b-e964e21c4a35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:31.776643 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b916db-7401-438b-8f4b-e964e21c4a35 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:31.794707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-c1b916db-7401-438b-8f4b-e964e21c4a35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:31.794707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:31.794707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:31.794707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:31.794707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:31.794707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:31.794707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:31.794707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:31.794707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:31.794707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:31.794707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:31.794707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:31.794707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:29:31.794707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:29:31.794707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:29:31.794707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:31.795802 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:29:31.796666 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:29:31.796666 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:31.796666 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:29:31.796666 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 472/942] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:31.802343 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879f7d0-533e-4c12-a93b-425d342c25ee None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:31.806294 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:31.918594 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879f7d0-533e-4c12-a93b-425d342c25ee None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:31.957115 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552c668-0895-4bf1-90f1-b8f8a3ea4d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:31.957788 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552c668-0895-4bf1-90f1-b8f8a3ea4d2b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:31.958062 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552c668-0895-4bf1-90f1-b8f8a3ea4d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:31.958330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552c668-0895-4bf1-90f1-b8f8a3ea4d2b None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:31.958969 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552c668-0895-4bf1-90f1-b8f8a3ea4d2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:31.959301 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552c668-0895-4bf1-90f1-b8f8a3ea4d2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:31.960723 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552c668-0895-4bf1-90f1-b8f8a3ea4d2b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:31.973007 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 471/943] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:31.979806 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b916db-7401-438b-8f4b-e964e21c4a35 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:32.028235 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:32.090556 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b916db-7401-438b-8f4b-e964e21c4a35 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:32.129924 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72325c64-1a2b-4ff2-bedc-2082dcf69da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:32.130569 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72325c64-1a2b-4ff2-bedc-2082dcf69da0 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:32.130844 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72325c64-1a2b-4ff2-bedc-2082dcf69da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:32.131112 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72325c64-1a2b-4ff2-bedc-2082dcf69da0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:32.138883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72325c64-1a2b-4ff2-bedc-2082dcf69da0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:32.140431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72325c64-1a2b-4ff2-bedc-2082dcf69da0 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:32.155665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-72325c64-1a2b-4ff2-bedc-2082dcf69da0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 12 14:29:32.155665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:32.155665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:32.155665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:32.155665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:32.155665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:32.155665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:32.155665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:32.155665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:32.155665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:32.155665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:32.155665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:32.155665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:29:32.155665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:29:32.155665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:29:32.155665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:32.156768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:32.157845 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:32.157845 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:32.157845 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:29:32.157845 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:29:32.157845 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:29:32.157845 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:29:32.157845 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:29:32.157845 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:29:32.157845 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 12 14:29:32.157845 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:29:32.158404 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 473/944] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:29:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:32.164632 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552c668-0895-4bf1-90f1-b8f8a3ea4d2b None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:32.233742 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:32.284140 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552c668-0895-4bf1-90f1-b8f8a3ea4d2b None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:32.326964 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ae001-b882-4b57-9747-d61c2a6bffad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:32.327738 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ae001-b882-4b57-9747-d61c2a6bffad None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:32.328025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ae001-b882-4b57-9747-d61c2a6bffad None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:32.328302 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ae001-b882-4b57-9747-d61c2a6bffad None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:32.329106 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ae001-b882-4b57-9747-d61c2a6bffad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:32.329411 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ae001-b882-4b57-9747-d61c2a6bffad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:32.330894 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ae001-b882-4b57-9747-d61c2a6bffad None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:32.350499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 472/945] 10.222.0.16 () {56 vars in 1143 bytes} [Wed Oct 12 14:29:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 14:29:32.357204 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72325c64-1a2b-4ff2-bedc-2082dcf69da0 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:32.473329 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72325c64-1a2b-4ff2-bedc-2082dcf69da0 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:32.474874 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:32.508188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66955647-0b08-4f4e-9327-d41f7e25d113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:32.508921 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66955647-0b08-4f4e-9327-d41f7e25d113 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:32.509249 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66955647-0b08-4f4e-9327-d41f7e25d113 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:32.509523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66955647-0b08-4f4e-9327-d41f7e25d113 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:32.522864 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66955647-0b08-4f4e-9327-d41f7e25d113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:32.524682 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66955647-0b08-4f4e-9327-d41f7e25d113 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:32.542234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-66955647-0b08-4f4e-9327-d41f7e25d113 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:32.542234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:32.542234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:32.542234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:32.542234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:32.542234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:32.542234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:32.542234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:32.542234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:32.542234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:32.542234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:32.542234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:32.542234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:29:32.542234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:29:32.542234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:29:32.542234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:32.543438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:29:32.544517 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:29:32.544517 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:32.544517 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:29:32.544517 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 474/946] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:29:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:32.549711 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ae001-b882-4b57-9747-d61c2a6bffad None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:32.661980 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ae001-b882-4b57-9747-d61c2a6bffad None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:32.685878 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:32.701520 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0325b-24b4-4789-bd2d-b8269f01d6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:32.702192 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0325b-24b4-4789-bd2d-b8269f01d6ff None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:32.702481 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0325b-24b4-4789-bd2d-b8269f01d6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:32.702756 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0325b-24b4-4789-bd2d-b8269f01d6ff None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:32.703511 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0325b-24b4-4789-bd2d-b8269f01d6ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:32.703854 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0325b-24b4-4789-bd2d-b8269f01d6ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:32.705291 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0325b-24b4-4789-bd2d-b8269f01d6ff None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:32.717494 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 473/947] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:32.723621 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66955647-0b08-4f4e-9327-d41f7e25d113 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:32.839499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66955647-0b08-4f4e-9327-d41f7e25d113 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:32.874996 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79fd4d3-f892-431f-8bac-8030d3813d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:32.875736 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fd4d3-f892-431f-8bac-8030d3813d96 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:32.876127 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fd4d3-f892-431f-8bac-8030d3813d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:32.876505 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fd4d3-f892-431f-8bac-8030d3813d96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:32.886468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79fd4d3-f892-431f-8bac-8030d3813d96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:32.888323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79fd4d3-f892-431f-8bac-8030d3813d96 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:32.902603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-a79fd4d3-f892-431f-8bac-8030d3813d96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:29:32.902603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:32.902603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:32.902603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:32.902603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:32.902603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:32.902603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:32.902603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:32.902603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:32.902603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:32.902603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:32.902603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:32.902603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:29:32.902603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:29:32.902603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:32.903613 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:32.904647 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:29:32.904647 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:29:32.904647 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:29:32.904647 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:29:32.904647 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:29:32.904647 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:29:32.904647 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:29:32.904647 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:29:32.904647 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 475/948] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:29:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:32.911324 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0325b-24b4-4789-bd2d-b8269f01d6ff None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:33.017858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0325b-24b4-4789-bd2d-b8269f01d6ff None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:33.058999 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4979d7-e1e1-4105-8e25-31a24c924eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:33.059631 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4979d7-e1e1-4105-8e25-31a24c924eab None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:33.059910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4979d7-e1e1-4105-8e25-31a24c924eab None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:33.060172 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4979d7-e1e1-4105-8e25-31a24c924eab None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:33.060856 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4979d7-e1e1-4105-8e25-31a24c924eab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:33.061155 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4979d7-e1e1-4105-8e25-31a24c924eab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:33.062701 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4979d7-e1e1-4105-8e25-31a24c924eab None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:33.075635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 474/949] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:29:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:33.081834 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79fd4d3-f892-431f-8bac-8030d3813d96 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:33.100041 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:33.196507 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79fd4d3-f892-431f-8bac-8030d3813d96 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:33.234400 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503faf4-3610-4422-a927-b74fbe666756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:33.235209 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503faf4-3610-4422-a927-b74fbe666756 None admin] REQUEST_METHOD: `PUT` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:33.235470 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503faf4-3610-4422-a927-b74fbe666756 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:33.235801 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503faf4-3610-4422-a927-b74fbe666756 None admin] PATH_INFO: `/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/93a6d6ccbf4940a9b292be17a94bfac8/roles/4b1f48ab911e4fadbc70164ae43d508b` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:33.237897 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503faf4-3610-4422-a927-b74fbe666756 None admin] RBAC: Authorizing `identity:create_grant(project_id=720b11237d2e48128c171be45fcd0ad0, user_id=93a6d6ccbf4940a9b292be17a94bfac8, role_id=4b1f48ab911e4fadbc70164ae43d508b)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:33.239558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503faf4-3610-4422-a927-b74fbe666756 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:33.254897 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 476/950] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Oct 12 14:29:33 2022] PUT /identity/v3/projects/720b11237d2e48128c171be45fcd0ad0/users/93a6d6ccbf4940a9b292be17a94bfac8/roles/4b1f48ab911e4fadbc70164ae43d508b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:29:35.714886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e5ce5-edce-4287-b4b1-61296f82497a None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:35.715532 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e5ce5-edce-4287-b4b1-61296f82497a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:35.715649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e5ce5-edce-4287-b4b1-61296f82497a None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:35.717248 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 475/951] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:29:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:29:35.722661 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89554cd2-97e6-43b2-a30e-df423b79ca03 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:35.723136 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89554cd2-97e6-43b2-a30e-df423b79ca03 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:35.723275 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89554cd2-97e6-43b2-a30e-df423b79ca03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:35.742721 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-89554cd2-97e6-43b2-a30e-df423b79ca03 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:29:35.743961 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-89554cd2-97e6-43b2-a30e-df423b79ca03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:29:35.819757 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 477/952] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:29:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:29:36.114164 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e5ce5-edce-4287-b4b1-61296f82497a None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:36.161503 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:36.226951 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e5ce5-edce-4287-b4b1-61296f82497a None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:36.269285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b24cb-dae4-4527-aff6-23ef083dfb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:36.270123 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b24cb-dae4-4527-aff6-23ef083dfb97 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:36.270495 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b24cb-dae4-4527-aff6-23ef083dfb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:36.270977 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b24cb-dae4-4527-aff6-23ef083dfb97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:36.284963 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b24cb-dae4-4527-aff6-23ef083dfb97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 14:29:36.290515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-749b24cb-dae4-4527-aff6-23ef083dfb97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:29:36.290515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:36.290515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:36.290515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:36.290515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:36.290515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:36.290515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:36.290515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:36.290515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:36.290515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:36.290515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:36.290515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:36.290515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 14:29:36.290515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 14:29:36.290515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 14:29:36.290515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 14:29:36.291635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 14:29:36.291635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 14:29:36.291635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 14:29:36.291635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 14:29:36.291635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:36.291635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:36.291635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:36.291635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:36.291635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:36.291635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:36.291635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:36.291635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:36.291635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:36.291635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:36.291635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:36.291635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:36.292462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:36.292462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:36.292462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:36.292462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:36.292462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 14:29:36.292462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 14:29:36.292462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 14:29:36.292462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 14:29:36.292462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 14:29:36.292462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 14:29:36.292462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 14:29:36.292462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:36.292462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 476/953] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Oct 12 14:29:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:36.297735 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89554cd2-97e6-43b2-a30e-df423b79ca03 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:36.301526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:36.402975 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89554cd2-97e6-43b2-a30e-df423b79ca03 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:36.438721 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3be3aa-29d8-40c7-97ea-2fdb583f41c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:36.439362 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3be3aa-29d8-40c7-97ea-2fdb583f41c0 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:36.439634 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3be3aa-29d8-40c7-97ea-2fdb583f41c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:36.439897 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3be3aa-29d8-40c7-97ea-2fdb583f41c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:36.440567 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3be3aa-29d8-40c7-97ea-2fdb583f41c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:36.440862 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3be3aa-29d8-40c7-97ea-2fdb583f41c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:36.442238 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3be3aa-29d8-40c7-97ea-2fdb583f41c0 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:36.452085 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 478/954] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Oct 12 14:29:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:36.457910 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b24cb-dae4-4527-aff6-23ef083dfb97 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:36.558569 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:36.561684 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b24cb-dae4-4527-aff6-23ef083dfb97 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:36.601811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5f72a-113d-4e52-b337-cf3a612391c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:36.602425 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5f72a-113d-4e52-b337-cf3a612391c6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:36.602693 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5f72a-113d-4e52-b337-cf3a612391c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:36.602956 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5f72a-113d-4e52-b337-cf3a612391c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:36.613380 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5f72a-113d-4e52-b337-cf3a612391c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:36.614796 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5f72a-113d-4e52-b337-cf3a612391c6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:36.629124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-39c5f72a-113d-4e52-b337-cf3a612391c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:36.629124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:36.629124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:36.629124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:36.629124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:36.629124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:36.629124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:36.629124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:36.629124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:36.629124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:36.629124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:36.629124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:36.629124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:29:36.629124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:29:36.629124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:29:36.629124 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:36.630203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:29:36.632065 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:29:36.632065 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:36.632065 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:36.632065 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 477/955] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:29:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:36.637437 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3be3aa-29d8-40c7-97ea-2fdb583f41c0 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:36.705341 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:36.742885 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3be3aa-29d8-40c7-97ea-2fdb583f41c0 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:36.775263 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f12c0f-615c-4ef2-a542-f4a62aa950da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:36.775858 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f12c0f-615c-4ef2-a542-f4a62aa950da None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:36.776131 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f12c0f-615c-4ef2-a542-f4a62aa950da None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:36.776396 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f12c0f-615c-4ef2-a542-f4a62aa950da None admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:36.777018 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f12c0f-615c-4ef2-a542-f4a62aa950da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:36.777312 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f12c0f-615c-4ef2-a542-f4a62aa950da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:36.778656 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f12c0f-615c-4ef2-a542-f4a62aa950da None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:36.790823 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 479/956] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:36.796521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5f72a-113d-4e52-b337-cf3a612391c6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:36.891673 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5f72a-113d-4e52-b337-cf3a612391c6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:36.924358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c07c9f-d0d0-4df3-8b2f-3bf4c0889370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:36.924967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c07c9f-d0d0-4df3-8b2f-3bf4c0889370 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:36.925241 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c07c9f-d0d0-4df3-8b2f-3bf4c0889370 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:36.925515 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c07c9f-d0d0-4df3-8b2f-3bf4c0889370 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:36.932616 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c07c9f-d0d0-4df3-8b2f-3bf4c0889370 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:36.934028 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c07c9f-d0d0-4df3-8b2f-3bf4c0889370 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:36.945368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-f2c07c9f-d0d0-4df3-8b2f-3bf4c0889370 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 12 14:29:36.945368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:36.945368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:36.945368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:36.945368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:36.945368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:36.945368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:36.945368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:36.945368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:36.945368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:36.945368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:36.945368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:36.945368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 14:29:36.945368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 14:29:36.945368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 14:29:36.945368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:36.946417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:36.947376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:36.947376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:36.947376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 14:29:36.947376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 14:29:36.947376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 14:29:36.947376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 14:29:36.947376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 14:29:36.947376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 14:29:36.947376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 12 14:29:36.947376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:36.947376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 478/957] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:29:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:36.952245 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f12c0f-615c-4ef2-a542-f4a62aa950da None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:37.048623 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f12c0f-615c-4ef2-a542-f4a62aa950da None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:37.073900 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:37.082539 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bff4c-da51-49e9-abf6-7cdaf2cf0990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:37.083124 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bff4c-da51-49e9-abf6-7cdaf2cf0990 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:37.083421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bff4c-da51-49e9-abf6-7cdaf2cf0990 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:37.083684 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bff4c-da51-49e9-abf6-7cdaf2cf0990 None admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:37.084441 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bff4c-da51-49e9-abf6-7cdaf2cf0990 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:37.084739 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bff4c-da51-49e9-abf6-7cdaf2cf0990 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:37.086173 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bff4c-da51-49e9-abf6-7cdaf2cf0990 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:37.103015 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 480/958] 10.222.0.16 () {56 vars in 1143 bytes} [Wed Oct 12 14:29:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:37.108728 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c07c9f-d0d0-4df3-8b2f-3bf4c0889370 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:37.124182 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:37.206302 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c07c9f-d0d0-4df3-8b2f-3bf4c0889370 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:37.246500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2c7bd-00b4-470b-8441-08db9c46ac59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:37.247198 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2c7bd-00b4-470b-8441-08db9c46ac59 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:37.247503 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2c7bd-00b4-470b-8441-08db9c46ac59 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:37.247802 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2c7bd-00b4-470b-8441-08db9c46ac59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:37.313253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2c7bd-00b4-470b-8441-08db9c46ac59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:37.314621 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2c7bd-00b4-470b-8441-08db9c46ac59 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:37.327465 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:37.408419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 479/959] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Oct 12 14:29:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 14:29:37.414792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bff4c-da51-49e9-abf6-7cdaf2cf0990 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:37.508765 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bff4c-da51-49e9-abf6-7cdaf2cf0990 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:37.542047 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e5be0-e028-48a4-a432-f13198a5b325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:37.542655 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e5be0-e028-48a4-a432-f13198a5b325 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:37.542926 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e5be0-e028-48a4-a432-f13198a5b325 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:37.543213 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e5be0-e028-48a4-a432-f13198a5b325 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:37.552766 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e5be0-e028-48a4-a432-f13198a5b325 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:37.554222 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e5be0-e028-48a4-a432-f13198a5b325 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:37.568186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-303e5be0-e028-48a4-a432-f13198a5b325 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:37.568186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:37.568186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:37.568186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:37.568186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:37.568186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:37.568186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:37.568186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:37.568186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:37.568186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:37.568186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:37.568186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:37.568186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 14:29:37.568186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 14:29:37.568186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 14:29:37.568186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:37.569558 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 14:29:37.570661 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 14:29:37.570661 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 14:29:37.570661 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:29:37.570661 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 481/960] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:29:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:37.575107 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2c7bd-00b4-470b-8441-08db9c46ac59 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:37.670314 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2c7bd-00b4-470b-8441-08db9c46ac59 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:37.683701 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:37.704099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c81592-3380-4c53-8426-2f0fcee3706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:37.704691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c81592-3380-4c53-8426-2f0fcee3706d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:37.704967 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c81592-3380-4c53-8426-2f0fcee3706d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:37.705235 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c81592-3380-4c53-8426-2f0fcee3706d None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:37.705861 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c81592-3380-4c53-8426-2f0fcee3706d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:37.706155 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c81592-3380-4c53-8426-2f0fcee3706d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:37.707528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c81592-3380-4c53-8426-2f0fcee3706d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:37.718694 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 480/961] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Oct 12 14:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:37.724380 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e5be0-e028-48a4-a432-f13198a5b325 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:37.759134 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:37.828344 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e5be0-e028-48a4-a432-f13198a5b325 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:37.861325 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58aaeae-4071-4280-81f6-183a948e24c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:37.861933 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58aaeae-4071-4280-81f6-183a948e24c1 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:37.862204 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58aaeae-4071-4280-81f6-183a948e24c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:37.862469 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58aaeae-4071-4280-81f6-183a948e24c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:37.869002 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58aaeae-4071-4280-81f6-183a948e24c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:37.870413 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58aaeae-4071-4280-81f6-183a948e24c1 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:37.881743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application [None req-a58aaeae-4071-4280-81f6-183a948e24c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:29:37.881743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:37.881743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:37.881743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:37.881743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:37.881743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:37.881743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:37.881743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:37.881743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:37.881743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:37.881743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:37.881743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:37.881743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 14:29:37.881743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 14:29:37.881743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:37.882792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:37.883777 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 14:29:37.883777 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 14:29:37.883777 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 14:29:37.883777 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 14:29:37.883777 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 14:29:37.883777 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 14:29:37.883777 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 14:29:37.883777 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: ERROR keystone.server.flask.application Oct 12 14:29:37.883777 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 482/962] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Oct 12 14:29:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:37.889008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c81592-3380-4c53-8426-2f0fcee3706d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:37.985894 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c81592-3380-4c53-8426-2f0fcee3706d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:38.018039 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7e018-ca78-4ff0-941a-c0e571aaaca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:38.018626 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7e018-ca78-4ff0-941a-c0e571aaaca8 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:38.018895 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7e018-ca78-4ff0-941a-c0e571aaaca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:38.019190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7e018-ca78-4ff0-941a-c0e571aaaca8 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:38.019861 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7e018-ca78-4ff0-941a-c0e571aaaca8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:38.020158 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7e018-ca78-4ff0-941a-c0e571aaaca8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:38.021555 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7e018-ca78-4ff0-941a-c0e571aaaca8 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:38.033245 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 481/963] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Oct 12 14:29:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:38.039897 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58aaeae-4071-4280-81f6-183a948e24c1 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:38.130628 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:38.137259 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58aaeae-4071-4280-81f6-183a948e24c1 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:38.169625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc204529-6233-4205-9a75-ee7d99efcc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:38.170206 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc204529-6233-4205-9a75-ee7d99efcc9e None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:38.170479 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc204529-6233-4205-9a75-ee7d99efcc9e None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:38.170744 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc204529-6233-4205-9a75-ee7d99efcc9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:38.171485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc204529-6233-4205-9a75-ee7d99efcc9e None admin] RBAC: Adding query filter params (role.id=4b1f48ab911e4fadbc70164ae43d508b, scope.project.id=720b11237d2e48128c171be45fcd0ad0, user.id=93a6d6ccbf4940a9b292be17a94bfac8) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:38.171788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc204529-6233-4205-9a75-ee7d99efcc9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:38.173197 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc204529-6233-4205-9a75-ee7d99efcc9e None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:38.179229 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 483/964] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Oct 12 14:29:38 2022] GET /identity/v3/role_assignments?user.id=93a6d6ccbf4940a9b292be17a94bfac8&scope.project.id=720b11237d2e48128c171be45fcd0ad0&role.id=4b1f48ab911e4fadbc70164ae43d508b => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:40.313631 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b94141-1cbc-4874-b78f-707929784f07 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:40.314162 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b94141-1cbc-4874-b78f-707929784f07 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:40.314162 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b94141-1cbc-4874-b78f-707929784f07 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:40.315516 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 482/965] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:29:40.321438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346eb5c-995f-4aa9-bffa-401e7482de61 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:40.321834 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346eb5c-995f-4aa9-bffa-401e7482de61 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:40.321982 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346eb5c-995f-4aa9-bffa-401e7482de61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:40.340415 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-e346eb5c-995f-4aa9-bffa-401e7482de61 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:29:40.341584 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-e346eb5c-995f-4aa9-bffa-401e7482de61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:29:40.406388 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 484/966] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:29:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:29:40.681219 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b94141-1cbc-4874-b78f-707929784f07 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:40.711860 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:40.779860 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b94141-1cbc-4874-b78f-707929784f07 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:40.812619 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a62f18-8b20-4dfc-8c4a-f928d467a6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:40.813228 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a62f18-8b20-4dfc-8c4a-f928d467a6ad None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:40.813505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a62f18-8b20-4dfc-8c4a-f928d467a6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:40.813776 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a62f18-8b20-4dfc-8c4a-f928d467a6ad None admin] PATH_INFO: `/v3/services/placement` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:40.814349 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a62f18-8b20-4dfc-8c4a-f928d467a6ad None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:40.815729 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a62f18-8b20-4dfc-8c4a-f928d467a6ad None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-16a62f18-8b20-4dfc-8c4a-f928d467a6ad None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:40.825068 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:40.826333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:40.826333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:40.826333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:40.826333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:40.826333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:40.826333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:40.826333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:29:40.826333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:29:40.826333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:29:40.826333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:29:40.826333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:40.826333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:40.826333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:40.826333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:40.826333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:40.826333 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:40.827459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:40.827459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:40.827459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:40.827459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:40.827459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:40.827459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:40.827459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:40.827459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:40.827459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:40.827459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:40.827459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:29:40.827459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:29:40.827459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 14:29:40.827459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:40.827459 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 483/967] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Oct 12 14:29:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:40.832176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346eb5c-995f-4aa9-bffa-401e7482de61 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:40.918288 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:40.927770 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346eb5c-995f-4aa9-bffa-401e7482de61 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:40.960216 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496946e9-c8ea-4ee8-be91-77f31dbffaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:40.960805 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496946e9-c8ea-4ee8-be91-77f31dbffaab None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:40.961077 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496946e9-c8ea-4ee8-be91-77f31dbffaab None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:40.961345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496946e9-c8ea-4ee8-be91-77f31dbffaab None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:40.961958 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496946e9-c8ea-4ee8-be91-77f31dbffaab None admin] RBAC: Adding query filter params (name=placement) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:40.962255 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496946e9-c8ea-4ee8-be91-77f31dbffaab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:40.963649 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496946e9-c8ea-4ee8-be91-77f31dbffaab None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:40.968247 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 485/968] 10.222.0.16 () {56 vars in 1116 bytes} [Wed Oct 12 14:29:40 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:40.973891 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a62f18-8b20-4dfc-8c4a-f928d467a6ad None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:41.065918 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:41.072008 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a62f18-8b20-4dfc-8c4a-f928d467a6ad None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:41.104079 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4c2ba-81d4-4f55-a15b-24efb1ee4b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:41.104615 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4c2ba-81d4-4f55-a15b-24efb1ee4b85 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:41.104892 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4c2ba-81d4-4f55-a15b-24efb1ee4b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:41.105163 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4c2ba-81d4-4f55-a15b-24efb1ee4b85 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:41.105783 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4c2ba-81d4-4f55-a15b-24efb1ee4b85 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:41.106082 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4c2ba-81d4-4f55-a15b-24efb1ee4b85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:41.107450 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4c2ba-81d4-4f55-a15b-24efb1ee4b85 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:41.111670 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 484/969] 10.222.0.16 () {56 vars in 1116 bytes} [Wed Oct 12 14:29:40 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:43.223879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d09e9-509a-40e7-b04e-9e8dd015761f None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:43.224597 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d09e9-509a-40e7-b04e-9e8dd015761f None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:43.224597 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d09e9-509a-40e7-b04e-9e8dd015761f None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:43.226144 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 486/970] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:29:43.231896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85292ad-4d78-46df-ba9b-d1572f7eb791 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:43.232342 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85292ad-4d78-46df-ba9b-d1572f7eb791 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:43.232535 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85292ad-4d78-46df-ba9b-d1572f7eb791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:43.252860 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-f85292ad-4d78-46df-ba9b-d1572f7eb791 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:29:43.254177 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f85292ad-4d78-46df-ba9b-d1572f7eb791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:29:43.323374 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 485/971] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:29:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:29:43.588878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d09e9-509a-40e7-b04e-9e8dd015761f None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:43.688330 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d09e9-509a-40e7-b04e-9e8dd015761f None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:43.720937 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552bf85-04f5-4707-b4f2-3da34e071cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:43.721528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552bf85-04f5-4707-b4f2-3da34e071cb0 None admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:43.721798 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552bf85-04f5-4707-b4f2-3da34e071cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:43.722059 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552bf85-04f5-4707-b4f2-3da34e071cb0 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:43.722701 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552bf85-04f5-4707-b4f2-3da34e071cb0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:43.724072 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552bf85-04f5-4707-b4f2-3da34e071cb0 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:43.737681 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 487/972] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Oct 12 14:29:43 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:29:45.871338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b18a7d-941b-444f-a52e-4f7342ffd320 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:45.871853 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b18a7d-941b-444f-a52e-4f7342ffd320 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:45.871853 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b18a7d-941b-444f-a52e-4f7342ffd320 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:45.873121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 486/973] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:29:45.878287 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00611577-12d0-4021-9198-527458aa7279 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:45.878660 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00611577-12d0-4021-9198-527458aa7279 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:45.878821 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00611577-12d0-4021-9198-527458aa7279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:45.897546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-00611577-12d0-4021-9198-527458aa7279 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:29:45.898704 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-00611577-12d0-4021-9198-527458aa7279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:29:45.907826 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:45.971053 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 488/974] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:29:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:29:46.239362 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b18a7d-941b-444f-a52e-4f7342ffd320 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:46.329932 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:46.340663 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b18a7d-941b-444f-a52e-4f7342ffd320 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:46.372220 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ff696-8026-4064-bf85-250b48abf9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:46.372832 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ff696-8026-4064-bf85-250b48abf9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:46.373102 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ff696-8026-4064-bf85-250b48abf9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:46.373363 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ff696-8026-4064-bf85-250b48abf9a4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:46.373928 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ff696-8026-4064-bf85-250b48abf9a4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:46.375300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ff696-8026-4064-bf85-250b48abf9a4 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-6a3ff696-8026-4064-bf85-250b48abf9a4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:46.384394 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:46.385537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:46.385537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:46.385537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:46.385537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:46.385537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:46.385537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:46.385537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:29:46.385537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:29:46.385537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:29:46.385537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:29:46.385537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:46.385537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:46.385537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:46.385537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:46.385537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:46.385537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:46.386528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:46.386528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:46.386528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:46.386528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:46.386528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:46.386528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:46.386528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:46.386528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:46.386528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:46.386528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:46.386528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:29:46.386528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:29:46.386528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 14:29:46.386528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:46.386528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 487/975] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Oct 12 14:29:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:46.391528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00611577-12d0-4021-9198-527458aa7279 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:46.487116 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00611577-12d0-4021-9198-527458aa7279 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:46.520289 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917eee3-2bf4-4bcf-8394-1db6a1679958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:46.520810 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917eee3-2bf4-4bcf-8394-1db6a1679958 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:46.521087 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917eee3-2bf4-4bcf-8394-1db6a1679958 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:46.521368 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917eee3-2bf4-4bcf-8394-1db6a1679958 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:46.522300 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917eee3-2bf4-4bcf-8394-1db6a1679958 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:46.522592 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917eee3-2bf4-4bcf-8394-1db6a1679958 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:46.523958 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917eee3-2bf4-4bcf-8394-1db6a1679958 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:46.528772 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 489/976] 10.222.0.16 () {56 vars in 1116 bytes} [Wed Oct 12 14:29:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:46.534876 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ff696-8026-4064-bf85-250b48abf9a4 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:46.632486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ff696-8026-4064-bf85-250b48abf9a4 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:46.664207 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f293b31-e557-492f-a63b-26a027b026b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:46.664793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f293b31-e557-492f-a63b-26a027b026b1 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:46.665063 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f293b31-e557-492f-a63b-26a027b026b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:46.665326 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f293b31-e557-492f-a63b-26a027b026b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:46.666004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f293b31-e557-492f-a63b-26a027b026b1 None admin] RBAC: Adding query filter params (interface=public, service_id=fb37e631e30f4fe9b34c07e5eaf5e6a4, region_id=RegionOne) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:46.666302 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f293b31-e557-492f-a63b-26a027b026b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:46.667676 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f293b31-e557-492f-a63b-26a027b026b1 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:46.672414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 488/977] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Oct 12 14:29:46 2022] GET /identity/v3/endpoints?service_id=fb37e631e30f4fe9b34c07e5eaf5e6a4&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:46.678515 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917eee3-2bf4-4bcf-8394-1db6a1679958 None admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:46.682222 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:46.774496 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917eee3-2bf4-4bcf-8394-1db6a1679958 None admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:46.807231 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264ad3c-16dd-42e6-80a8-a0782cc6cdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:46.807900 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264ad3c-16dd-42e6-80a8-a0782cc6cdf6 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:46.808181 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264ad3c-16dd-42e6-80a8-a0782cc6cdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:46.808450 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264ad3c-16dd-42e6-80a8-a0782cc6cdf6 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:46.809041 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264ad3c-16dd-42e6-80a8-a0782cc6cdf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:46.810385 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264ad3c-16dd-42e6-80a8-a0782cc6cdf6 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:46.817014 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 490/978] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Oct 12 14:29:46 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 14:29:48.931356 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17083cc-5c5f-4316-bbe0-2e7e39e9bd4f None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:48.931893 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17083cc-5c5f-4316-bbe0-2e7e39e9bd4f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:48.931893 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17083cc-5c5f-4316-bbe0-2e7e39e9bd4f None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:48.933192 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 489/979] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:29:48.938342 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47476397-970a-4af4-9aef-3776b1d6815a None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:48.938744 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47476397-970a-4af4-9aef-3776b1d6815a None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:48.938857 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47476397-970a-4af4-9aef-3776b1d6815a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:48.957269 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-47476397-970a-4af4-9aef-3776b1d6815a None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:29:48.958418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-47476397-970a-4af4-9aef-3776b1d6815a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:29:49.017968 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:49.024199 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 491/980] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:29:48 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 14:29:49.298167 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17083cc-5c5f-4316-bbe0-2e7e39e9bd4f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:49.317494 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:49.396327 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17083cc-5c5f-4316-bbe0-2e7e39e9bd4f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:49.428072 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66839a7-9237-4a87-bd25-909fa704ec35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:49.428609 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66839a7-9237-4a87-bd25-909fa704ec35 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:49.428881 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66839a7-9237-4a87-bd25-909fa704ec35 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:49.429144 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66839a7-9237-4a87-bd25-909fa704ec35 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:49.429709 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66839a7-9237-4a87-bd25-909fa704ec35 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:49.431040 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66839a7-9237-4a87-bd25-909fa704ec35 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-e66839a7-9237-4a87-bd25-909fa704ec35 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 14:29:49.439581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 14:29:49.440747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 14:29:49.440747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 14:29:49.440747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 14:29:49.440747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 14:29:49.440747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 14:29:49.440747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 14:29:49.440747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 14:29:49.440747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 14:29:49.440747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 14:29:49.440747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 14:29:49.440747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 14:29:49.440747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 14:29:49.440747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 14:29:49.440747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 14:29:49.440747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 14:29:49.440747 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 14:29:49.441598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 14:29:49.441598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 12 14:29:49.441598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 14:29:49.441598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 12 14:29:49.441598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 14:29:49.441598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 14:29:49.441598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 14:29:49.441598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 12 14:29:49.441598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 14:29:49.441598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 12 14:29:49.441598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 14:29:49.441598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 14:29:49.441598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 14:29:49.441598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 12 14:29:49.441598 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 490/981] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Oct 12 14:29:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 14:29:49.447129 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47476397-970a-4af4-9aef-3776b1d6815a None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:49.542360 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47476397-970a-4af4-9aef-3776b1d6815a None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:49.575793 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529bc94-4cd2-459d-bd7d-0403cd684e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:49.576327 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529bc94-4cd2-459d-bd7d-0403cd684e30 None admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:49.576603 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529bc94-4cd2-459d-bd7d-0403cd684e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:49.576871 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529bc94-4cd2-459d-bd7d-0403cd684e30 None admin] PATH_INFO: `/v3/services` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:49.577489 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529bc94-4cd2-459d-bd7d-0403cd684e30 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:29:49.577916 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529bc94-4cd2-459d-bd7d-0403cd684e30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:49.579312 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529bc94-4cd2-459d-bd7d-0403cd684e30 None admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:49.584735 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 492/982] 10.222.0.16 () {56 vars in 1116 bytes} [Wed Oct 12 14:29:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:29:49.590340 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66839a7-9237-4a87-bd25-909fa704ec35 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:29:49.668840 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:49.687085 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66839a7-9237-4a87-bd25-909fa704ec35 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:29:49.719552 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db613e-9911-495b-aae8-fce99eb94f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:29:49.720143 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db613e-9911-495b-aae8-fce99eb94f07 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:29:49.720421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db613e-9911-495b-aae8-fce99eb94f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:29:49.720692 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db613e-9911-495b-aae8-fce99eb94f07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:29:49.721361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db613e-9911-495b-aae8-fce99eb94f07 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:29:49.722710 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db613e-9911-495b-aae8-fce99eb94f07 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:29:49.741637 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 491/983] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Oct 12 14:29:49 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:29:59.581580 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=69 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:29:59.736950 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:30:54.368669 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d6cdc-a101-4d44-bffd-6420909e938e None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:30:54.369167 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d6cdc-a101-4d44-bffd-6420909e938e None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:30:54.369321 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d6cdc-a101-4d44-bffd-6420909e938e None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:30:54.370949 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 493/984] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:30:54.381305 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19cf33-3159-4862-b2c6-e987df4052bc None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:30:54.381918 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19cf33-3159-4862-b2c6-e987df4052bc None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:30:54.381918 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19cf33-3159-4862-b2c6-e987df4052bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:30:54.552337 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3a19cf33-3159-4862-b2c6-e987df4052bc None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:30:54.554453 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3a19cf33-3159-4862-b2c6-e987df4052bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:30:54.693210 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 492/985] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:30:55.009133 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee1870-4b71-4c2f-bfca-2effad827ce7 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:30:55.009451 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee1870-4b71-4c2f-bfca-2effad827ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:30:55.009667 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee1870-4b71-4c2f-bfca-2effad827ce7 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:30:55.011242 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 494/986] 10.222.0.16 () {54 vars in 964 bytes} [Wed Oct 12 14:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:30:55.017753 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fac0e-9adf-431e-86d7-200973fb39aa None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:30:55.018027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fac0e-9adf-431e-86d7-200973fb39aa None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:30:55.018299 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fac0e-9adf-431e-86d7-200973fb39aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:30:55.109351 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a60fac0e-9adf-431e-86d7-200973fb39aa None None] MFA Rules not processed for user `9d0d533e36ff4a5186f8704480b58646`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:30:55.110631 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a60fac0e-9adf-431e-86d7-200973fb39aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:30:55.133363 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:30:55.205356 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee1870-4b71-4c2f-bfca-2effad827ce7 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:30:55.207105 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-02ee1870-4b71-4c2f-bfca-2effad827ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:30:55.211353 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 493/987] 10.222.0.16 () {58 vars in 1061 bytes} [Wed Oct 12 14:30:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 14:30:55.328568 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee1870-4b71-4c2f-bfca-2effad827ce7 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:30:55.414349 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1206c-9727-4996-bf75-faeeff8a3b70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:30:55.415750 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1206c-9727-4996-bf75-faeeff8a3b70 service neutron] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:30:55.416171 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1206c-9727-4996-bf75-faeeff8a3b70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:30:55.416581 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1206c-9727-4996-bf75-faeeff8a3b70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:30:55.418323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-97c1206c-9727-4996-bf75-faeeff8a3b70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:30:55.478674 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:30:55.542617 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1206c-9727-4996-bf75-faeeff8a3b70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:30:55.544328 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1206c-9727-4996-bf75-faeeff8a3b70 service neutron] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:30:55.564538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 495/988] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:30:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:30:58.104594 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1931e-3bcd-4a38-8c79-62a0c238b3c1 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:30:58.105096 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1931e-3bcd-4a38-8c79-62a0c238b3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:30:58.105096 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1931e-3bcd-4a38-8c79-62a0c238b3c1 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:30:58.106363 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 494/989] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:30:58.111681 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e620ab8-f2cb-4b7a-81a9-a9f1ba3735c8 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:30:58.112101 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e620ab8-f2cb-4b7a-81a9-a9f1ba3735c8 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:30:58.112219 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e620ab8-f2cb-4b7a-81a9-a9f1ba3735c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:30:58.182170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-3e620ab8-f2cb-4b7a-81a9-a9f1ba3735c8 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:30:58.183376 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-3e620ab8-f2cb-4b7a-81a9-a9f1ba3735c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:30:58.278144 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 496/990] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:30:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:30:58.559921 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1931e-3bcd-4a38-8c79-62a0c238b3c1 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:30:58.561427 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6be1931e-3bcd-4a38-8c79-62a0c238b3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:30:58.665764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1931e-3bcd-4a38-8c79-62a0c238b3c1 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:30:58.736935 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:30:58.739225 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda4906-c7fc-4960-af4e-bc05bf76d977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:30:58.739872 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda4906-c7fc-4960-af4e-bc05bf76d977 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:30:58.740367 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda4906-c7fc-4960-af4e-bc05bf76d977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:30:58.740816 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda4906-c7fc-4960-af4e-bc05bf76d977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:30:58.742653 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-deda4906-c7fc-4960-af4e-bc05bf76d977 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:30:58.850552 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda4906-c7fc-4960-af4e-bc05bf76d977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:30:58.851934 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda4906-c7fc-4960-af4e-bc05bf76d977 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:30:58.869753 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 495/991] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:30:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:31:01.316264 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7317aeb-f679-4511-b57d-c5cc6d9bc2d2 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:01.316792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7317aeb-f679-4511-b57d-c5cc6d9bc2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:01.316792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7317aeb-f679-4511-b57d-c5cc6d9bc2d2 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:01.318124 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 497/992] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:01.323376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab6b43-9bb5-463a-92e5-586f3a73a7d5 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:01.323813 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab6b43-9bb5-463a-92e5-586f3a73a7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:01.323907 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab6b43-9bb5-463a-92e5-586f3a73a7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:01.388316 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-5fab6b43-9bb5-463a-92e5-586f3a73a7d5 None None] MFA Rules not processed for user `0b1931fa42bd48ffae39f00dd53304fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:01.389516 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-5fab6b43-9bb5-463a-92e5-586f3a73a7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:01.441031 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:01.485612 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 496/993] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:31:01 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:01.768313 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7317aeb-f679-4511-b57d-c5cc6d9bc2d2 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:31:01.769823 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-a7317aeb-f679-4511-b57d-c5cc6d9bc2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:01.779821 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:01.872374 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7317aeb-f679-4511-b57d-c5cc6d9bc2d2 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:31:01.944109 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7958b-2a04-4b4d-b738-f543a55ecf7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:31:01.944700 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7958b-2a04-4b4d-b738-f543a55ecf7f service neutron] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:01.944968 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7958b-2a04-4b4d-b738-f543a55ecf7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:01.945236 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7958b-2a04-4b4d-b738-f543a55ecf7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:01.946776 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-06a7958b-2a04-4b4d-b738-f543a55ecf7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:02.007762 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:02.053839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7958b-2a04-4b4d-b738-f543a55ecf7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:31:02.055294 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7958b-2a04-4b4d-b738-f543a55ecf7f service neutron] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:31:02.073479 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 498/994] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:31:01 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:31:04.747521 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf3f4f-8700-418a-b2c6-2621552c6d25 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:04.748066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf3f4f-8700-418a-b2c6-2621552c6d25 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:04.748066 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf3f4f-8700-418a-b2c6-2621552c6d25 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:04.749445 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 497/995] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:04.754803 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b68ac-bb08-493e-85fa-cada9bc85ea2 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:04.755247 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b68ac-bb08-493e-85fa-cada9bc85ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:04.755387 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b68ac-bb08-493e-85fa-cada9bc85ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:04.821358 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-d86b68ac-bb08-493e-85fa-cada9bc85ea2 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:04.822524 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-d86b68ac-bb08-493e-85fa-cada9bc85ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:04.916588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 499/996] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:31:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:05.197951 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf3f4f-8700-418a-b2c6-2621552c6d25 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:31:05.199505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-aeaf3f4f-8700-418a-b2c6-2621552c6d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:05.300796 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf3f4f-8700-418a-b2c6-2621552c6d25 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:31:05.371548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c9ca6-67c2-4281-ae31-e6521776d009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:31:05.372145 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c9ca6-67c2-4281-ae31-e6521776d009 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:05.372419 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c9ca6-67c2-4281-ae31-e6521776d009 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:05.372686 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c9ca6-67c2-4281-ae31-e6521776d009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:05.374810 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-437c9ca6-67c2-4281-ae31-e6521776d009 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:05.381624 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:05.481911 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c9ca6-67c2-4281-ae31-e6521776d009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:31:05.483284 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c9ca6-67c2-4281-ae31-e6521776d009 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:31:05.501045 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 498/997] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:31:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:31:08.298206 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d973e8-bb8c-41e8-a091-78fa2a609749 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:08.298732 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d973e8-bb8c-41e8-a091-78fa2a609749 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:08.298732 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d973e8-bb8c-41e8-a091-78fa2a609749 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:08.300252 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 500/998] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:08.305484 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d9c98-c457-42b2-98dc-a26d2dee929a None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:08.305909 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d9c98-c457-42b2-98dc-a26d2dee929a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:08.306044 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d9c98-c457-42b2-98dc-a26d2dee929a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:08.378903 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a15d9c98-c457-42b2-98dc-a26d2dee929a None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:08.380173 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a15d9c98-c457-42b2-98dc-a26d2dee929a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:08.443491 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:08.487695 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 499/999] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:08.773994 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d973e8-bb8c-41e8-a091-78fa2a609749 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:31:08.775593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-99d973e8-bb8c-41e8-a091-78fa2a609749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:08.796442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:08.879421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d973e8-bb8c-41e8-a091-78fa2a609749 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:31:08.952842 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89ec02-658c-4031-bbbb-122f34a1f1da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:31:08.953546 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89ec02-658c-4031-bbbb-122f34a1f1da service neutron] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:08.953830 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89ec02-658c-4031-bbbb-122f34a1f1da service neutron] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:08.954101 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89ec02-658c-4031-bbbb-122f34a1f1da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:08.955709 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-ec89ec02-658c-4031-bbbb-122f34a1f1da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:09.027969 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:09.066563 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89ec02-658c-4031-bbbb-122f34a1f1da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:31:09.068035 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89ec02-658c-4031-bbbb-122f34a1f1da service neutron] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:31:09.086218 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 501/1000] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:31:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:31:11.301669 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6655b99-7cd9-4a8a-bd58-949faeb8f895 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:11.302195 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6655b99-7cd9-4a8a-bd58-949faeb8f895 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:11.302195 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6655b99-7cd9-4a8a-bd58-949faeb8f895 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:11.303587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 500/1001] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:11.308797 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06318081-99ff-4804-b176-adad7ef3d5ea None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:11.309182 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06318081-99ff-4804-b176-adad7ef3d5ea None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:11.309332 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06318081-99ff-4804-b176-adad7ef3d5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:11.377019 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-06318081-99ff-4804-b176-adad7ef3d5ea None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:11.378191 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-06318081-99ff-4804-b176-adad7ef3d5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:11.481818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 502/1002] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:11.769326 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6655b99-7cd9-4a8a-bd58-949faeb8f895 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:31:11.770835 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c6655b99-7cd9-4a8a-bd58-949faeb8f895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:11.872264 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6655b99-7cd9-4a8a-bd58-949faeb8f895 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:31:11.947182 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181443a5-757f-429b-8959-7707edbd4f46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:31:11.947897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181443a5-757f-429b-8959-7707edbd4f46 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:11.948172 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181443a5-757f-429b-8959-7707edbd4f46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:11.948436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181443a5-757f-429b-8959-7707edbd4f46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:11.950004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-181443a5-757f-429b-8959-7707edbd4f46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:11.967059 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:12.061498 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181443a5-757f-429b-8959-7707edbd4f46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:31:12.062843 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181443a5-757f-429b-8959-7707edbd4f46 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:31:12.080994 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 501/1003] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:31:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:31:14.279109 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1d5ed-a1c2-4083-a188-7aa742be25b9 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:14.280293 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1d5ed-a1c2-4083-a188-7aa742be25b9 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:14.280499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1d5ed-a1c2-4083-a188-7aa742be25b9 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:14.281913 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 503/1004] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:31:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:14.287430 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7ef64-f39b-428d-a079-51b9797cc75c None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:14.287887 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7ef64-f39b-428d-a079-51b9797cc75c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:14.287998 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7ef64-f39b-428d-a079-51b9797cc75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:14.352603 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-41b7ef64-f39b-428d-a079-51b9797cc75c None None] MFA Rules not processed for user `0b1931fa42bd48ffae39f00dd53304fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:14.353774 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-41b7ef64-f39b-428d-a079-51b9797cc75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:14.419458 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:14.452153 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 502/1005] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:31:14 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:14.738930 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1d5ed-a1c2-4083-a188-7aa742be25b9 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:31:14.740502 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-46b1d5ed-a1c2-4083-a188-7aa742be25b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:14.773176 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:14.851077 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1d5ed-a1c2-4083-a188-7aa742be25b9 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:31:14.925064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc973844-15dd-4787-a95b-1148d7e7cbfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:31:14.925667 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc973844-15dd-4787-a95b-1148d7e7cbfd service neutron] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:14.925938 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc973844-15dd-4787-a95b-1148d7e7cbfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:14.926204 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc973844-15dd-4787-a95b-1148d7e7cbfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:14.927799 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-fc973844-15dd-4787-a95b-1148d7e7cbfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:15.005330 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:15.038016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc973844-15dd-4787-a95b-1148d7e7cbfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:31:15.039477 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc973844-15dd-4787-a95b-1148d7e7cbfd service neutron] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:31:15.057888 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 504/1006] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:31:14 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:31:17.444606 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966864b-e365-479b-a7c3-f3cc98d9ded2 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:17.445141 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966864b-e365-479b-a7c3-f3cc98d9ded2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:17.445141 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966864b-e365-479b-a7c3-f3cc98d9ded2 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:17.446455 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 503/1007] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:17.452262 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03e929-4491-418d-8167-480c51b63555 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:17.452691 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03e929-4491-418d-8167-480c51b63555 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:17.452784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03e929-4491-418d-8167-480c51b63555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:17.517093 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-ee03e929-4491-418d-8167-480c51b63555 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:17.518266 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-ee03e929-4491-418d-8167-480c51b63555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:17.613669 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 505/1008] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:17.899811 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966864b-e365-479b-a7c3-f3cc98d9ded2 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:31:17.901592 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9966864b-e365-479b-a7c3-f3cc98d9ded2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:18.003530 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966864b-e365-479b-a7c3-f3cc98d9ded2 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:31:18.075855 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcd760-e188-4cd5-a466-6c2eca640007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:31:18.077078 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd760-e188-4cd5-a466-6c2eca640007 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:18.077351 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd760-e188-4cd5-a466-6c2eca640007 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:18.077619 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd760-e188-4cd5-a466-6c2eca640007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:18.079202 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-39bcd760-e188-4cd5-a466-6c2eca640007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:18.105479 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:18.185373 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcd760-e188-4cd5-a466-6c2eca640007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:31:18.187486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcd760-e188-4cd5-a466-6c2eca640007 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:31:18.206081 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 504/1009] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:31:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:31:20.536239 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5a39e-db9d-430c-a1a1-0d3ca6526328 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:20.536816 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5a39e-db9d-430c-a1a1-0d3ca6526328 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:20.536816 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5a39e-db9d-430c-a1a1-0d3ca6526328 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:20.538113 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 506/1010] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:20.543404 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdb927-5baf-4356-a922-ee09eb3e72d6 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:20.543797 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdb927-5baf-4356-a922-ee09eb3e72d6 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:20.543977 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdb927-5baf-4356-a922-ee09eb3e72d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:20.610213 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-0ebdb927-5baf-4356-a922-ee09eb3e72d6 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:20.611425 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0ebdb927-5baf-4356-a922-ee09eb3e72d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:20.684252 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:20.706266 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 505/1011] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:20.995495 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5a39e-db9d-430c-a1a1-0d3ca6526328 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:31:20.997037 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-63f5a39e-db9d-430c-a1a1-0d3ca6526328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:21.037655 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:21.100431 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5a39e-db9d-430c-a1a1-0d3ca6526328 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:31:21.171952 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3246c89-219d-4e4e-84c7-65400c24a6bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:31:21.172474 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3246c89-219d-4e4e-84c7-65400c24a6bc service neutron] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:21.172743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3246c89-219d-4e4e-84c7-65400c24a6bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:21.173008 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3246c89-219d-4e4e-84c7-65400c24a6bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:21.174556 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-e3246c89-219d-4e4e-84c7-65400c24a6bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:21.258377 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:21.284412 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3246c89-219d-4e4e-84c7-65400c24a6bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:31:21.285844 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3246c89-219d-4e4e-84c7-65400c24a6bc service neutron] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:31:21.304098 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 507/1012] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:31:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:31:23.758615 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cb56c-e816-4438-8400-a3c34ed7382c None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:23.759220 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cb56c-e816-4438-8400-a3c34ed7382c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:23.759220 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cb56c-e816-4438-8400-a3c34ed7382c None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:23.760545 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 506/1013] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:23.766417 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf67493-c7fc-4d79-af09-01e1c68b21fb None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:23.766836 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf67493-c7fc-4d79-af09-01e1c68b21fb None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:23.766927 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf67493-c7fc-4d79-af09-01e1c68b21fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:23.832788 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-2bf67493-c7fc-4d79-af09-01e1c68b21fb None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:23.833971 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-2bf67493-c7fc-4d79-af09-01e1c68b21fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:23.927425 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 508/1014] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:31:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:24.212405 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cb56c-e816-4438-8400-a3c34ed7382c None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:31:24.213935 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-1d3cb56c-e816-4438-8400-a3c34ed7382c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:24.319707 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cb56c-e816-4438-8400-a3c34ed7382c None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:31:24.392162 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180aa02-33df-457b-bd05-8aae52800225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:31:24.392763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180aa02-33df-457b-bd05-8aae52800225 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:24.393035 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180aa02-33df-457b-bd05-8aae52800225 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:24.393301 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180aa02-33df-457b-bd05-8aae52800225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:24.394868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f180aa02-33df-457b-bd05-8aae52800225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:24.430145 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:24.503292 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180aa02-33df-457b-bd05-8aae52800225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:31:24.504629 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180aa02-33df-457b-bd05-8aae52800225 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:31:24.523043 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 507/1015] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:31:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:31:28.602455 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b920ed-8068-42cc-b647-31601b88e62a None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:28.602927 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b920ed-8068-42cc-b647-31601b88e62a None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:28.602927 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b920ed-8068-42cc-b647-31601b88e62a None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:28.604298 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 509/1016] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:28.609707 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e35abb-4c20-4189-8a23-a752d4936bd2 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:28.610147 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e35abb-4c20-4189-8a23-a752d4936bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:28.610251 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e35abb-4c20-4189-8a23-a752d4936bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:28.678147 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-24e35abb-4c20-4189-8a23-a752d4936bd2 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:28.679377 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-24e35abb-4c20-4189-8a23-a752d4936bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:28.765300 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:28.777258 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 508/1017] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:29.061694 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b920ed-8068-42cc-b647-31601b88e62a None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:31:29.063241 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-45b920ed-8068-42cc-b647-31601b88e62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:29.112091 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:29.170167 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b920ed-8068-42cc-b647-31601b88e62a None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:31:29.245818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2258b40-02ba-402d-81a7-b60ba0f6408e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:31:29.246418 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2258b40-02ba-402d-81a7-b60ba0f6408e service neutron] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:29.246727 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2258b40-02ba-402d-81a7-b60ba0f6408e service neutron] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:29.246998 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2258b40-02ba-402d-81a7-b60ba0f6408e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:29.248705 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-e2258b40-02ba-402d-81a7-b60ba0f6408e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:29.342878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:29.359974 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2258b40-02ba-402d-81a7-b60ba0f6408e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:31:29.361411 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2258b40-02ba-402d-81a7-b60ba0f6408e service neutron] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:31:29.379551 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 510/1018] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:31:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:31:32.212173 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c3858-cb20-49b9-8d00-b0b73b23341c None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:32.212704 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c3858-cb20-49b9-8d00-b0b73b23341c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:32.212704 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c3858-cb20-49b9-8d00-b0b73b23341c None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:32.214031 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 509/1019] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:32.219538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08336f6-b9fd-4e67-b0a9-b6fac43b0cb4 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:32.219998 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08336f6-b9fd-4e67-b0a9-b6fac43b0cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:32.220144 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08336f6-b9fd-4e67-b0a9-b6fac43b0cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:32.292182 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-b08336f6-b9fd-4e67-b0a9-b6fac43b0cb4 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:32.293889 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-b08336f6-b9fd-4e67-b0a9-b6fac43b0cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:32.396649 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 511/1020] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:31:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:32.682507 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c3858-cb20-49b9-8d00-b0b73b23341c None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:31:32.684129 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c00c3858-cb20-49b9-8d00-b0b73b23341c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:32.788023 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c3858-cb20-49b9-8d00-b0b73b23341c None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:31:32.862181 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac4777e-ac0c-45d5-addf-eeeab3bf70ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:31:32.862709 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4777e-ac0c-45d5-addf-eeeab3bf70ac service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:32.862983 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4777e-ac0c-45d5-addf-eeeab3bf70ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:32.863284 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4777e-ac0c-45d5-addf-eeeab3bf70ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:32.864862 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-fac4777e-ac0c-45d5-addf-eeeab3bf70ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:32.910430 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:32.975900 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac4777e-ac0c-45d5-addf-eeeab3bf70ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:31:32.977316 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac4777e-ac0c-45d5-addf-eeeab3bf70ac service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:31:32.995629 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 510/1021] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:31:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:31:37.175846 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c1903-b7f6-4e69-a0c3-1eaa48455ff8 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:37.176303 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c1903-b7f6-4e69-a0c3-1eaa48455ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:37.176404 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c1903-b7f6-4e69-a0c3-1eaa48455ff8 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:37.177738 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 512/1022] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:37.183110 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57962fd0-2ad6-4cd8-9301-74bb73d1e944 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:37.184230 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57962fd0-2ad6-4cd8-9301-74bb73d1e944 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:37.184424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57962fd0-2ad6-4cd8-9301-74bb73d1e944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:37.253852 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-57962fd0-2ad6-4cd8-9301-74bb73d1e944 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:37.255079 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-57962fd0-2ad6-4cd8-9301-74bb73d1e944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:37.354268 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:37.358317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 511/1023] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:37.643195 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c1903-b7f6-4e69-a0c3-1eaa48455ff8 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:31:37.644739 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-c97c1903-b7f6-4e69-a0c3-1eaa48455ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:37.701956 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:37.748404 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c1903-b7f6-4e69-a0c3-1eaa48455ff8 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:31:37.827041 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677f18a-6ae3-44ba-9121-56711a36b6fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:31:37.827668 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677f18a-6ae3-44ba-9121-56711a36b6fa service neutron] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:37.827936 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677f18a-6ae3-44ba-9121-56711a36b6fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:37.828200 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677f18a-6ae3-44ba-9121-56711a36b6fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:37.829779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-9677f18a-6ae3-44ba-9121-56711a36b6fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:37.938127 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:37.942590 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677f18a-6ae3-44ba-9121-56711a36b6fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:31:37.944166 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677f18a-6ae3-44ba-9121-56711a36b6fa service neutron] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:31:37.963017 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 513/1024] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:31:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:31:42.902748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addadbc-9b81-4420-9b33-44dc7944eb9e None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:42.903726 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addadbc-9b81-4420-9b33-44dc7944eb9e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:42.904149 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addadbc-9b81-4420-9b33-44dc7944eb9e None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:42.905912 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 512/1025] 10.222.0.16 () {54 vars in 921 bytes} [Wed Oct 12 14:31:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:42.912142 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c9343-6704-43fe-a3ff-40a7ffa36f73 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:42.912562 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c9343-6704-43fe-a3ff-40a7ffa36f73 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:42.912954 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c9343-6704-43fe-a3ff-40a7ffa36f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:43.035189 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-884c9343-6704-43fe-a3ff-40a7ffa36f73 None None] MFA Rules not processed for user `93a6d6ccbf4940a9b292be17a94bfac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:43.038897 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-884c9343-6704-43fe-a3ff-40a7ffa36f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:43.200301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 514/1026] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Oct 12 14:31:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:47.326651 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da1dcf-0e86-4678-89b0-4c399b5bcb98 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:47.327266 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da1dcf-0e86-4678-89b0-4c399b5bcb98 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:47.327266 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da1dcf-0e86-4678-89b0-4c399b5bcb98 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:47.328742 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 513/1027] 10.222.0.16 () {54 vars in 921 bytes} [Wed Oct 12 14:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:47.333792 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e57f3b-1ef6-48f1-b928-a006df1c765a None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:47.334617 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e57f3b-1ef6-48f1-b928-a006df1c765a None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:47.335119 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e57f3b-1ef6-48f1-b928-a006df1c765a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:47.392072 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:47.422624 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-a6e57f3b-1ef6-48f1-b928-a006df1c765a None None] MFA Rules not processed for user `93a6d6ccbf4940a9b292be17a94bfac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:47.424127 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-a6e57f3b-1ef6-48f1-b928-a006df1c765a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:47.542669 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 515/1028] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Oct 12 14:31:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:49.108498 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6120b7-bd27-46ba-9a03-e9271f4ce018 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:49.109015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6120b7-bd27-46ba-9a03-e9271f4ce018 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:49.109015 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6120b7-bd27-46ba-9a03-e9271f4ce018 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:49.110358 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 514/1029] 10.222.0.16 () {54 vars in 921 bytes} [Wed Oct 12 14:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:49.115503 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7143c4-a1c3-4db6-b96c-7770ef3457b4 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:49.115914 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7143c4-a1c3-4db6-b96c-7770ef3457b4 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:49.116043 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7143c4-a1c3-4db6-b96c-7770ef3457b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:49.192567 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-ef7143c4-a1c3-4db6-b96c-7770ef3457b4 None None] MFA Rules not processed for user `93a6d6ccbf4940a9b292be17a94bfac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:49.193812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-ef7143c4-a1c3-4db6-b96c-7770ef3457b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:49.236117 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:49.310172 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 516/1030] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Oct 12 14:31:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:52.753732 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981fd10-9d12-475a-8e49-025eb4b0225e None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:52.754205 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981fd10-9d12-475a-8e49-025eb4b0225e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:52.754284 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981fd10-9d12-475a-8e49-025eb4b0225e None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:52.755943 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 515/1031] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:52.762512 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4dd56-579d-4ee4-ac04-8ce31e83e379 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:52.763275 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4dd56-579d-4ee4-ac04-8ce31e83e379 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:52.763275 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4dd56-579d-4ee4-ac04-8ce31e83e379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:52.836523 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-b6d4dd56-579d-4ee4-ac04-8ce31e83e379 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:52.837591 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-b6d4dd56-579d-4ee4-ac04-8ce31e83e379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:52.938945 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 517/1032] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:53.243323 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08670c7-0239-4c5e-8fd4-3438d72b4e24 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:53.244851 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08670c7-0239-4c5e-8fd4-3438d72b4e24 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:53.245528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08670c7-0239-4c5e-8fd4-3438d72b4e24 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:53.247475 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 516/1033] 10.222.0.16 () {54 vars in 961 bytes} [Wed Oct 12 14:31:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:53.256505 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87eb16-ef25-414d-af86-f984535f5f4f None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:53.257432 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87eb16-ef25-414d-af86-f984535f5f4f None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:53.258045 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87eb16-ef25-414d-af86-f984535f5f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:53.277677 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:53.347850 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-2e87eb16-ef25-414d-af86-f984535f5f4f None None] MFA Rules not processed for user `d0aeed5f32a1477dbd9d3e934669705b`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:53.349516 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-2e87eb16-ef25-414d-af86-f984535f5f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:53.476263 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 518/1034] 10.222.0.16 () {58 vars in 1058 bytes} [Wed Oct 12 14:31:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:53.484062 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08670c7-0239-4c5e-8fd4-3438d72b4e24 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:31:53.485874 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c08670c7-0239-4c5e-8fd4-3438d72b4e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:53.657417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08670c7-0239-4c5e-8fd4-3438d72b4e24 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:31:53.733874 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:53.776209 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619da6a3-eb4e-447d-a9d3-b9bc49d131c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:31:53.777004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619da6a3-eb4e-447d-a9d3-b9bc49d131c2 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:53.777282 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619da6a3-eb4e-447d-a9d3-b9bc49d131c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:53.777550 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619da6a3-eb4e-447d-a9d3-b9bc49d131c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:53.779438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-619da6a3-eb4e-447d-a9d3-b9bc49d131c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:53.880969 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619da6a3-eb4e-447d-a9d3-b9bc49d131c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:31:53.882682 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619da6a3-eb4e-447d-a9d3-b9bc49d131c2 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:31:53.902244 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 517/1035] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:31:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 14:31:56.293168 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa6f85-e32e-47f3-a358-eda270d001a9 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:56.293664 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa6f85-e32e-47f3-a358-eda270d001a9 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:56.293664 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa6f85-e32e-47f3-a358-eda270d001a9 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:56.295188 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 519/1036] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:31:56.300844 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c8815-8ff6-4cb5-9b73-854ed5359f83 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:56.301431 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c8815-8ff6-4cb5-9b73-854ed5359f83 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:56.301431 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c8815-8ff6-4cb5-9b73-854ed5359f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:56.379305 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-2f2c8815-8ff6-4cb5-9b73-854ed5359f83 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:56.380498 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2f2c8815-8ff6-4cb5-9b73-854ed5359f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:56.421923 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:56.487436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 518/1037] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:31:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:31:56.779702 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa6f85-e32e-47f3-a358-eda270d001a9 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:31:56.781353 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-35aa6f85-e32e-47f3-a358-eda270d001a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:56.857164 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:56.961142 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa6f85-e32e-47f3-a358-eda270d001a9 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:31:57.086931 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a277eb-4d39-4f07-9e3e-134d8a5ad141 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:31:57.087588 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a277eb-4d39-4f07-9e3e-134d8a5ad141 service nova] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:57.087865 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a277eb-4d39-4f07-9e3e-134d8a5ad141 service nova] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:57.088135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a277eb-4d39-4f07-9e3e-134d8a5ad141 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:57.089844 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-20a277eb-4d39-4f07-9e3e-134d8a5ad141 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:57.114590 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:57.186790 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a277eb-4d39-4f07-9e3e-134d8a5ad141 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:31:57.188408 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a277eb-4d39-4f07-9e3e-134d8a5ad141 service nova] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:31:57.206767 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 520/1038] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:31:59.718397 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc8171-eeea-40c1-8e2e-9c26ede220c7 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:59.718934 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc8171-eeea-40c1-8e2e-9c26ede220c7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:59.718934 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc8171-eeea-40c1-8e2e-9c26ede220c7 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:59.720436 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 519/1039] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:31:59.725467 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed651442-9d34-4604-9b42-d1e11d68e75d None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:31:59.725900 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed651442-9d34-4604-9b42-d1e11d68e75d None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:31:59.725994 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed651442-9d34-4604-9b42-d1e11d68e75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:31:59.796875 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-ed651442-9d34-4604-9b42-d1e11d68e75d None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:31:59.798562 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-ed651442-9d34-4604-9b42-d1e11d68e75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:31:59.897354 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:31:59.907466 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 521/1040] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:00.195336 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7b944-8dde-4d8e-9339-8f653946876f None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:00.195717 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7b944-8dde-4d8e-9339-8f653946876f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:00.196078 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7b944-8dde-4d8e-9339-8f653946876f None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:00.198254 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 520/1041] 10.222.0.16 () {54 vars in 961 bytes} [Wed Oct 12 14:32:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:00.203818 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cca43-1965-4390-bbc2-00db1cd9ab6f None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:00.204094 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cca43-1965-4390-bbc2-00db1cd9ab6f None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:00.204357 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cca43-1965-4390-bbc2-00db1cd9ab6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:00.285185 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-d29cca43-1965-4390-bbc2-00db1cd9ab6f None None] MFA Rules not processed for user `d0aeed5f32a1477dbd9d3e934669705b`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:00.286357 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-d29cca43-1965-4390-bbc2-00db1cd9ab6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:00.407324 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 522/1042] 10.222.0.16 () {58 vars in 1058 bytes} [Wed Oct 12 14:32:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:00.415507 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7b944-8dde-4d8e-9339-8f653946876f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:00.417172 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c4d7b944-8dde-4d8e-9339-8f653946876f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:00.576179 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7b944-8dde-4d8e-9339-8f653946876f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:00.598113 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:00.699978 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d987799-5a39-49f5-ae10-4e228612a983 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:00.700585 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d987799-5a39-49f5-ae10-4e228612a983 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:00.700858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d987799-5a39-49f5-ae10-4e228612a983 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:00.701121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d987799-5a39-49f5-ae10-4e228612a983 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:00.702683 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6d987799-5a39-49f5-ae10-4e228612a983 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:00.798085 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d987799-5a39-49f5-ae10-4e228612a983 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:00.800195 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d987799-5a39-49f5-ae10-4e228612a983 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:00.823398 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 521/1043] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:00.956755 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e0477-afbd-425a-b308-40d27d6ec62f None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:00.957353 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e0477-afbd-425a-b308-40d27d6ec62f None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:00.957353 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e0477-afbd-425a-b308-40d27d6ec62f None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:00.958871 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 523/1044] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:00.965012 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e37144-3fed-46e6-9b54-6d24b8e8bcf7 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:00.965371 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e37144-3fed-46e6-9b54-6d24b8e8bcf7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:00.965573 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e37144-3fed-46e6-9b54-6d24b8e8bcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:01.005307 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:01.050005 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-68e37144-3fed-46e6-9b54-6d24b8e8bcf7 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:01.051428 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-68e37144-3fed-46e6-9b54-6d24b8e8bcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:01.164487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 522/1045] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:01.522346 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15aa1b-d6b8-45b2-a4e7-23c5edb9c688 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:01.523082 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15aa1b-d6b8-45b2-a4e7-23c5edb9c688 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:01.523568 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15aa1b-d6b8-45b2-a4e7-23c5edb9c688 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:01.525964 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 524/1046] 10.222.0.16 () {54 vars in 963 bytes} [Wed Oct 12 14:32:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:01.531217 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cf245-1067-43e1-ad37-c169b92d5afb None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:01.531568 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cf245-1067-43e1-ad37-c169b92d5afb None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:01.531925 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cf245-1067-43e1-ad37-c169b92d5afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:01.624787 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ef5cf245-1067-43e1-ad37-c169b92d5afb None None] MFA Rules not processed for user `c8514f660bf949ada15e8d7fc6a062a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:01.625222 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ef5cf245-1067-43e1-ad37-c169b92d5afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:01.679925 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:01.713253 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 523/1047] 10.222.0.16 () {58 vars in 1060 bytes} [Wed Oct 12 14:32:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:01.721541 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd15aa1b-d6b8-45b2-a4e7-23c5edb9c688 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:01.723076 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-cd15aa1b-d6b8-45b2-a4e7-23c5edb9c688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:01.779276 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:01.840750 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd15aa1b-d6b8-45b2-a4e7-23c5edb9c688 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:01.935775 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363d3b1-d374-48f8-98e0-fd844b7c6723 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8514f660bf949ada15e8d7fc6a062a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:01.936369 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363d3b1-d374-48f8-98e0-fd844b7c6723 service cinder] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:01.936743 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363d3b1-d374-48f8-98e0-fd844b7c6723 service cinder] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:01.936932 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363d3b1-d374-48f8-98e0-fd844b7c6723 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:01.938473 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-f363d3b1-d374-48f8-98e0-fd844b7c6723 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:02.122549 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:02.150184 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363d3b1-d374-48f8-98e0-fd844b7c6723 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:02.153472 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363d3b1-d374-48f8-98e0-fd844b7c6723 service cinder] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:02.215032 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 525/1048] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:03.069627 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b44e3-3817-40e8-bea2-e4de5ef37083 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:03.070176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b44e3-3817-40e8-bea2-e4de5ef37083 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:03.070176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b44e3-3817-40e8-bea2-e4de5ef37083 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:03.071541 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 524/1049] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:03.076935 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b5ffe-05f0-40ba-9050-a1503d4e6623 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:03.077221 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b5ffe-05f0-40ba-9050-a1503d4e6623 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:03.077494 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b5ffe-05f0-40ba-9050-a1503d4e6623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:03.142738 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-108b5ffe-05f0-40ba-9050-a1503d4e6623 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:03.143882 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-108b5ffe-05f0-40ba-9050-a1503d4e6623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:03.242248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 526/1050] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:03.530548 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b44e3-3817-40e8-bea2-e4de5ef37083 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:03.532167 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a07b44e3-3817-40e8-bea2-e4de5ef37083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:03.690541 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b44e3-3817-40e8-bea2-e4de5ef37083 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:03.751826 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:03.822608 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40666599-fd7f-44a5-8a7f-9a8eebf2492c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:03.823285 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40666599-fd7f-44a5-8a7f-9a8eebf2492c service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:03.823561 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40666599-fd7f-44a5-8a7f-9a8eebf2492c service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:03.823829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40666599-fd7f-44a5-8a7f-9a8eebf2492c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:03.825438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-40666599-fd7f-44a5-8a7f-9a8eebf2492c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:03.919111 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40666599-fd7f-44a5-8a7f-9a8eebf2492c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:03.920897 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40666599-fd7f-44a5-8a7f-9a8eebf2492c service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:03.939853 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 525/1051] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:06.171854 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20759a15-b0e5-4eb2-ab68-40c4b6f94e96 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:06.172872 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20759a15-b0e5-4eb2-ab68-40c4b6f94e96 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:06.173364 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20759a15-b0e5-4eb2-ab68-40c4b6f94e96 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:06.174922 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 527/1052] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:32:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:06.180798 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9ab4f-6268-4ac3-9424-2a6015ee81cb None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:06.181399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9ab4f-6268-4ac3-9424-2a6015ee81cb None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:06.181399 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9ab4f-6268-4ac3-9424-2a6015ee81cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:06.254421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-63e9ab4f-6268-4ac3-9424-2a6015ee81cb None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:06.255654 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-63e9ab4f-6268-4ac3-9424-2a6015ee81cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:06.263701 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:06.361923 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 526/1053] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:06.653587 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20759a15-b0e5-4eb2-ab68-40c4b6f94e96 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:06.655233 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-20759a15-b0e5-4eb2-ab68-40c4b6f94e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:06.720421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:06.815296 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20759a15-b0e5-4eb2-ab68-40c4b6f94e96 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:06.940264 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368238b5-e3e4-42c4-ad3b-d2496ae1f861 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:06.941268 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368238b5-e3e4-42c4-ad3b-d2496ae1f861 service nova] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:06.941646 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368238b5-e3e4-42c4-ad3b-d2496ae1f861 service nova] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:06.942018 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368238b5-e3e4-42c4-ad3b-d2496ae1f861 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:06.943735 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-368238b5-e3e4-42c4-ad3b-d2496ae1f861 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:06.964887 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:07.041402 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368238b5-e3e4-42c4-ad3b-d2496ae1f861 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:07.043164 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368238b5-e3e4-42c4-ad3b-d2496ae1f861 service nova] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:07.063380 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 528/1054] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:09.324547 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32570116-0266-4c70-8fa3-181a97945ff2 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:09.325081 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32570116-0266-4c70-8fa3-181a97945ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:09.325081 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32570116-0266-4c70-8fa3-181a97945ff2 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:09.326435 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 527/1055] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:09.332291 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d192c-68a3-4823-946e-f99b33cd7a3c None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:09.332710 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d192c-68a3-4823-946e-f99b33cd7a3c None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:09.332884 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d192c-68a3-4823-946e-f99b33cd7a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:09.402971 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-882d192c-68a3-4823-946e-f99b33cd7a3c None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:09.404239 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-882d192c-68a3-4823-946e-f99b33cd7a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:09.501036 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:09.507889 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 529/1056] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:09.801866 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32570116-0266-4c70-8fa3-181a97945ff2 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:09.803406 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-32570116-0266-4c70-8fa3-181a97945ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:09.944438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:09.956062 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32570116-0266-4c70-8fa3-181a97945ff2 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:10.077121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775f6f9-d5d6-4ec2-9230-9e954923f55c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:10.077733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775f6f9-d5d6-4ec2-9230-9e954923f55c service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:10.078010 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775f6f9-d5d6-4ec2-9230-9e954923f55c service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:10.078283 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775f6f9-d5d6-4ec2-9230-9e954923f55c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:10.079906 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c775f6f9-d5d6-4ec2-9230-9e954923f55c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:10.171448 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775f6f9-d5d6-4ec2-9230-9e954923f55c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:10.173074 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775f6f9-d5d6-4ec2-9230-9e954923f55c service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:10.183857 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:10.192323 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 528/1057] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:12.388090 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06013a9-bc99-433d-ac19-e7e662d6c88c None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:12.388639 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06013a9-bc99-433d-ac19-e7e662d6c88c None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:12.388639 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06013a9-bc99-433d-ac19-e7e662d6c88c None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:12.389963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 530/1058] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:12.396460 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925c1d0-afbf-465b-a288-1e204f16b8df None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:12.397176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925c1d0-afbf-465b-a288-1e204f16b8df None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:12.397272 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925c1d0-afbf-465b-a288-1e204f16b8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:12.466784 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-7925c1d0-afbf-465b-a288-1e204f16b8df None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:12.468033 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7925c1d0-afbf-465b-a288-1e204f16b8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:12.566865 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 529/1059] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:12.959038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06013a9-bc99-433d-ac19-e7e662d6c88c None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:12.960675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-d06013a9-bc99-433d-ac19-e7e662d6c88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:13.115111 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06013a9-bc99-433d-ac19-e7e662d6c88c None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:13.192779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:13.241611 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614437f1-b411-4904-bed7-81038681d32e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:13.242715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614437f1-b411-4904-bed7-81038681d32e service nova] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:13.242990 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614437f1-b411-4904-bed7-81038681d32e service nova] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:13.243285 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614437f1-b411-4904-bed7-81038681d32e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:13.245571 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-614437f1-b411-4904-bed7-81038681d32e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:13.337934 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614437f1-b411-4904-bed7-81038681d32e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:13.339564 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614437f1-b411-4904-bed7-81038681d32e service nova] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:13.358256 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 531/1060] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:16.174593 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b701ff-48e6-4e36-8bf2-8024c715298b None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:16.175206 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b701ff-48e6-4e36-8bf2-8024c715298b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:16.175206 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b701ff-48e6-4e36-8bf2-8024c715298b None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:16.176577 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 530/1061] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:16.183816 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d987c3-cc90-4096-9c25-dc5bacdfa0dc None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:16.184268 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d987c3-cc90-4096-9c25-dc5bacdfa0dc None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:16.184499 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d987c3-cc90-4096-9c25-dc5bacdfa0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:16.265155 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-05d987c3-cc90-4096-9c25-dc5bacdfa0dc None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:16.266358 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-05d987c3-cc90-4096-9c25-dc5bacdfa0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:16.300780 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:16.369157 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 532/1062] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:16.665522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b701ff-48e6-4e36-8bf2-8024c715298b None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:16.667055 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-b0b701ff-48e6-4e36-8bf2-8024c715298b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:16.744306 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:16.813872 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b701ff-48e6-4e36-8bf2-8024c715298b None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:16.932273 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f7652-1817-4a51-9419-832a20c76570 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:16.932880 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f7652-1817-4a51-9419-832a20c76570 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:16.933155 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f7652-1817-4a51-9419-832a20c76570 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:16.933418 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f7652-1817-4a51-9419-832a20c76570 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:16.934992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-e98f7652-1817-4a51-9419-832a20c76570 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:16.975748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:17.029062 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f7652-1817-4a51-9419-832a20c76570 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:17.030651 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f7652-1817-4a51-9419-832a20c76570 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:17.048235 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 531/1063] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:18.444747 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ae434-4da7-4462-9187-6c5e470404ef None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:18.445322 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ae434-4da7-4462-9187-6c5e470404ef None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:18.445322 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ae434-4da7-4462-9187-6c5e470404ef None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:18.446671 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 533/1064] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:18.452234 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4e5ac-cdb9-4c9a-8f98-99edfa8293c2 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:18.452657 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4e5ac-cdb9-4c9a-8f98-99edfa8293c2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:18.452764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4e5ac-cdb9-4c9a-8f98-99edfa8293c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:18.518145 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-cfb4e5ac-cdb9-4c9a-8f98-99edfa8293c2 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:18.519356 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-cfb4e5ac-cdb9-4c9a-8f98-99edfa8293c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:18.621800 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 532/1065] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:18.931940 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d2ea3-18df-43fa-9856-8acbfd8ba98f None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:18.932214 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d2ea3-18df-43fa-9856-8acbfd8ba98f None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:18.932480 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d2ea3-18df-43fa-9856-8acbfd8ba98f None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:18.933803 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 534/1066] 10.222.0.16 () {54 vars in 962 bytes} [Wed Oct 12 14:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:18.939138 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0133fd6-beb8-4863-be8f-ca5135ee06fb None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:18.939546 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0133fd6-beb8-4863-be8f-ca5135ee06fb None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:18.939827 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0133fd6-beb8-4863-be8f-ca5135ee06fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:18.955627 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:19.017625 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c0133fd6-beb8-4863-be8f-ca5135ee06fb None None] MFA Rules not processed for user `2b95d3ba7df041aab3281129f6516d20`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:19.018830 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c0133fd6-beb8-4863-be8f-ca5135ee06fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:19.095733 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 533/1067] 10.222.0.16 () {58 vars in 1059 bytes} [Wed Oct 12 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 14:32:19.105989 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d2ea3-18df-43fa-9856-8acbfd8ba98f None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:19.107577 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-a35d2ea3-18df-43fa-9856-8acbfd8ba98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:19.218698 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d2ea3-18df-43fa-9856-8acbfd8ba98f None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:19.261078 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db836dae-1754-46bc-93f6-ebcc3403a3cb None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:19.261361 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db836dae-1754-46bc-93f6-ebcc3403a3cb None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:19.261637 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db836dae-1754-46bc-93f6-ebcc3403a3cb None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:19.262979 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 534/1068] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 14:32:19.267994 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca913f3-8380-4b87-ae15-0a05fcb22f1b None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:19.268276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca913f3-8380-4b87-ae15-0a05fcb22f1b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:19.268843 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca913f3-8380-4b87-ae15-0a05fcb22f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:19.280836 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:19.298521 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab4e28-2f27-465f-8ff4-026d1cd7141b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b95d3ba7df041aab3281129f6516d20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:19.299121 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab4e28-2f27-465f-8ff4-026d1cd7141b service glance] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:19.299422 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab4e28-2f27-465f-8ff4-026d1cd7141b service glance] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:19.299685 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab4e28-2f27-465f-8ff4-026d1cd7141b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:19.301248 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-3fab4e28-2f27-465f-8ff4-026d1cd7141b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:19.342441 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-7ca913f3-8380-4b87-ae15-0a05fcb22f1b None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:19.343660 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7ca913f3-8380-4b87-ae15-0a05fcb22f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:19.399748 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:19.417509 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab4e28-2f27-465f-8ff4-026d1cd7141b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:19.418986 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab4e28-2f27-465f-8ff4-026d1cd7141b service glance] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:19.438416 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 535/1069] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:19.443179 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 535/1070] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 14:32:19.737274 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab4e28-2f27-465f-8ff4-026d1cd7141b service glance] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:19.738467 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-3fab4e28-2f27-465f-8ff4-026d1cd7141b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:19.855222 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:19.863253 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab4e28-2f27-465f-8ff4-026d1cd7141b service glance] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:19.989532 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec749f-54e6-4c92-b4f5-e937538dca4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:19.990144 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec749f-54e6-4c92-b4f5-e937538dca4f service nova] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:19.990416 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec749f-54e6-4c92-b4f5-e937538dca4f service nova] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:19.990680 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec749f-54e6-4c92-b4f5-e937538dca4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:19.993128 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-93ec749f-54e6-4c92-b4f5-e937538dca4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:20.094638 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec749f-54e6-4c92-b4f5-e937538dca4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:20.096593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec749f-54e6-4c92-b4f5-e937538dca4f service nova] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:20.112099 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:20.115676 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 536/1071] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:20.217114 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ede67-375d-413a-a285-9c89721fa485 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:20.217655 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ede67-375d-413a-a285-9c89721fa485 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:20.217655 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ede67-375d-413a-a285-9c89721fa485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:20.301708 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-cb3ede67-375d-413a-a285-9c89721fa485 None None] MFA Rules not processed for user `15d7cb817428475e80cf697ac94c62b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:20.302937 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-cb3ede67-375d-413a-a285-9c89721fa485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:20.402786 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 536/1072] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Oct 12 14:32:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:20.450038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a74eae-a56c-4a13-8b3f-15ad79e779a8 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:20.450450 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a74eae-a56c-4a13-8b3f-15ad79e779a8 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:20.450826 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a74eae-a56c-4a13-8b3f-15ad79e779a8 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:20.452526 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 537/1073] 10.222.0.16 () {54 vars in 943 bytes} [Wed Oct 12 14:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:20.459553 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa7875-20fd-4b6a-aa50-25a4c8730227 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:20.461009 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa7875-20fd-4b6a-aa50-25a4c8730227 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:20.461009 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa7875-20fd-4b6a-aa50-25a4c8730227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:20.526663 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:20.546809 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-75fa7875-20fd-4b6a-aa50-25a4c8730227 None None] MFA Rules not processed for user `c8381a7c961047c69de894cf5b0055b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:20.548176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-75fa7875-20fd-4b6a-aa50-25a4c8730227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:20.668012 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 537/1074] 10.222.0.16 () {58 vars in 1040 bytes} [Wed Oct 12 14:32:20 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:20.678967 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a74eae-a56c-4a13-8b3f-15ad79e779a8 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:20.680525 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-07a74eae-a56c-4a13-8b3f-15ad79e779a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:20.838644 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a74eae-a56c-4a13-8b3f-15ad79e779a8 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:20.934202 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:20.958663 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ef87f-99ed-4770-8257-24acc70e8c4d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8381a7c961047c69de894cf5b0055b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:20.960566 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ef87f-99ed-4770-8257-24acc70e8c4d service swift] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:20.960566 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ef87f-99ed-4770-8257-24acc70e8c4d service swift] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:20.960566 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ef87f-99ed-4770-8257-24acc70e8c4d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:20.961658 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-147ef87f-99ed-4770-8257-24acc70e8c4d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:21.055919 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ef87f-99ed-4770-8257-24acc70e8c4d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:21.057556 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ef87f-99ed-4770-8257-24acc70e8c4d service swift] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:21.068777 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 538/1075] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Oct 12 14:32:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 14:32:22.499255 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983f074-ceb7-4d19-b43f-81c801286ca9 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:22.499255 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983f074-ceb7-4d19-b43f-81c801286ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:22.499255 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983f074-ceb7-4d19-b43f-81c801286ca9 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:22.503303 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 538/1076] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:32:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:22.512246 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff3db-3f50-4389-b47b-b078e10beb59 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:22.512784 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff3db-3f50-4389-b47b-b078e10beb59 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:22.513246 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff3db-3f50-4389-b47b-b078e10beb59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:22.621170 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-720ff3db-3f50-4389-b47b-b078e10beb59 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:22.622530 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-720ff3db-3f50-4389-b47b-b078e10beb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:22.712729 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:22.751270 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 539/1077] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:23.056446 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983f074-ceb7-4d19-b43f-81c801286ca9 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:23.058666 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f983f074-ceb7-4d19-b43f-81c801286ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:23.200464 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:23.276716 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983f074-ceb7-4d19-b43f-81c801286ca9 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:23.449233 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7992f9-229a-409e-b250-22f466e7d2b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:23.451006 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7992f9-229a-409e-b250-22f466e7d2b7 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:23.451458 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7992f9-229a-409e-b250-22f466e7d2b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:23.451737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7992f9-229a-409e-b250-22f466e7d2b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:23.453417 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-5e7992f9-229a-409e-b250-22f466e7d2b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:23.558787 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:23.586948 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7992f9-229a-409e-b250-22f466e7d2b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:23.588951 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7992f9-229a-409e-b250-22f466e7d2b7 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:23.613696 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 539/1078] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:25.891950 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d300e-7b15-436b-bdee-659fc8ffb2c0 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:25.892473 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d300e-7b15-436b-bdee-659fc8ffb2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:25.892473 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d300e-7b15-436b-bdee-659fc8ffb2c0 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:25.893739 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 540/1079] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:25.899640 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f6270-8b12-48c5-b268-882dd0161075 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:25.900123 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f6270-8b12-48c5-b268-882dd0161075 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:25.900233 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f6270-8b12-48c5-b268-882dd0161075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:25.978210 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-710f6270-8b12-48c5-b268-882dd0161075 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:25.979452 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-710f6270-8b12-48c5-b268-882dd0161075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:26.142409 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 540/1080] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:26.447436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d300e-7b15-436b-bdee-659fc8ffb2c0 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:26.448938 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-6b5d300e-7b15-436b-bdee-659fc8ffb2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:26.643922 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d300e-7b15-436b-bdee-659fc8ffb2c0 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:26.725737 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:26.785542 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612baaa-38be-40bf-ad8b-c13fe5883f07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:26.786561 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612baaa-38be-40bf-ad8b-c13fe5883f07 service nova] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:26.786897 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612baaa-38be-40bf-ad8b-c13fe5883f07 service nova] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:26.787323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612baaa-38be-40bf-ad8b-c13fe5883f07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:26.790666 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-0612baaa-38be-40bf-ad8b-c13fe5883f07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:26.905073 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0612baaa-38be-40bf-ad8b-c13fe5883f07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:26.906696 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0612baaa-38be-40bf-ad8b-c13fe5883f07 service nova] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:26.935675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 541/1081] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:29.287773 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e477e1-3e0a-46c4-9237-b55272a95ad7 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:29.288813 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e477e1-3e0a-46c4-9237-b55272a95ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:29.289317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e477e1-3e0a-46c4-9237-b55272a95ad7 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:29.290886 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 541/1082] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:32:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:29.296391 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f15e2-278a-4582-bbef-b8c126b34ec2 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:29.296835 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f15e2-278a-4582-bbef-b8c126b34ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:29.296917 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f15e2-278a-4582-bbef-b8c126b34ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:29.381463 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-628f15e2-278a-4582-bbef-b8c126b34ec2 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:29.382831 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-628f15e2-278a-4582-bbef-b8c126b34ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:29.406098 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:29.509164 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 542/1083] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:29.799713 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e477e1-3e0a-46c4-9237-b55272a95ad7 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:29.801480 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-64e477e1-3e0a-46c4-9237-b55272a95ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:29.882828 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:30.008493 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e477e1-3e0a-46c4-9237-b55272a95ad7 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:30.159059 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250cba51-c13f-4fe5-93d1-7684f407b6e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:30.159820 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cba51-c13f-4fe5-93d1-7684f407b6e7 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:30.160156 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cba51-c13f-4fe5-93d1-7684f407b6e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:30.160486 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cba51-c13f-4fe5-93d1-7684f407b6e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:30.163541 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-250cba51-c13f-4fe5-93d1-7684f407b6e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:30.173308 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:30.267073 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250cba51-c13f-4fe5-93d1-7684f407b6e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:30.268783 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250cba51-c13f-4fe5-93d1-7684f407b6e7 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:30.289791 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 542/1084] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:39.507126 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=37 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:40.286646 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=41 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:45.242000 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70d748-7de5-4f2d-a1ec-e3d64b213f2e None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:45.242545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70d748-7de5-4f2d-a1ec-e3d64b213f2e None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:45.242545 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70d748-7de5-4f2d-a1ec-e3d64b213f2e None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:45.243922 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 543/1085] 10.222.0.16 () {54 vars in 918 bytes} [Wed Oct 12 14:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:45.248932 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6696c25-0fb5-4f93-ad54-7c542691882d None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:45.249368 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6696c25-0fb5-4f93-ad54-7c542691882d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:45.249472 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6696c25-0fb5-4f93-ad54-7c542691882d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:45.325574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c6696c25-0fb5-4f93-ad54-7c542691882d None None] MFA Rules not processed for user `93a6d6ccbf4940a9b292be17a94bfac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:45.326760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c6696c25-0fb5-4f93-ad54-7c542691882d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:45.430922 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 543/1086] 10.222.0.16 () {58 vars in 1015 bytes} [Wed Oct 12 14:32:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:45.445497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfd986-1d4b-4e7a-b801-e8dcdec08f5e None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:45.445772 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfd986-1d4b-4e7a-b801-e8dcdec08f5e None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:45.446039 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfd986-1d4b-4e7a-b801-e8dcdec08f5e None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:45.447450 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 544/1087] 10.222.0.16 () {54 vars in 918 bytes} [Wed Oct 12 14:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:45.452190 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba251a-ce45-499c-982b-08dfe8d4332d None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:45.452473 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba251a-ce45-499c-982b-08dfe8d4332d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:45.452769 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba251a-ce45-499c-982b-08dfe8d4332d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:45.524099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:45.527001 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-bdba251a-ce45-499c-982b-08dfe8d4332d None None] MFA Rules not processed for user `d0aeed5f32a1477dbd9d3e934669705b`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:45.528750 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-bdba251a-ce45-499c-982b-08dfe8d4332d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:45.642767 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 544/1088] 10.222.0.16 () {58 vars in 1015 bytes} [Wed Oct 12 14:32:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:48.223178 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e657a-a814-45f4-ae87-862a2cf77846 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:48.223781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e657a-a814-45f4-ae87-862a2cf77846 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:48.223781 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e657a-a814-45f4-ae87-862a2cf77846 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:48.225062 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 545/1089] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 14:32:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:48.230393 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152bd8b-7884-48d4-9089-4fb1432b1a82 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:48.230822 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152bd8b-7884-48d4-9089-4fb1432b1a82 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:48.230965 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152bd8b-7884-48d4-9089-4fb1432b1a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:48.298301 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-7152bd8b-7884-48d4-9089-4fb1432b1a82 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:48.299531 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7152bd8b-7884-48d4-9089-4fb1432b1a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:48.354375 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:48.395915 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 545/1090] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 14:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 14:32:48.684177 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e657a-a814-45f4-ae87-862a2cf77846 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:48.685680 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-529e657a-a814-45f4-ae87-862a2cf77846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:48.832166 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e657a-a814-45f4-ae87-862a2cf77846 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:48.903093 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:48.944088 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3dc9e-b701-46b6-be32-9c0f9e41881d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:48.944694 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3dc9e-b701-46b6-be32-9c0f9e41881d service nova] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:48.944960 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3dc9e-b701-46b6-be32-9c0f9e41881d service nova] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:48.945223 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3dc9e-b701-46b6-be32-9c0f9e41881d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:48.946787 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-48f3dc9e-b701-46b6-be32-9c0f9e41881d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:49.036289 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3dc9e-b701-46b6-be32-9c0f9e41881d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:49.037862 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3dc9e-b701-46b6-be32-9c0f9e41881d service nova] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:49.056064 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 546/1091] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:50.185121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d8132-f439-4c22-b779-19294e7fa93f None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:50.185719 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d8132-f439-4c22-b779-19294e7fa93f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:50.185719 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d8132-f439-4c22-b779-19294e7fa93f None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:50.186977 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 546/1092] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 14:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:50.193837 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4062c49-e406-4d9a-ac63-d0162bd8aecf None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:50.194257 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4062c49-e406-4d9a-ac63-d0162bd8aecf None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:50.194370 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4062c49-e406-4d9a-ac63-d0162bd8aecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:50.269165 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-c4062c49-e406-4d9a-ac63-d0162bd8aecf None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:50.270342 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-c4062c49-e406-4d9a-ac63-d0162bd8aecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:50.297103 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:50.365667 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 547/1093] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 14:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:50.515370 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d8132-f439-4c22-b779-19294e7fa93f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:50.516780 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-127d8132-f439-4c22-b779-19294e7fa93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:50.662245 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d8132-f439-4c22-b779-19294e7fa93f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:50.702105 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:50.772581 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f3078-05db-433e-8c17-244c238c4d2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:50.773103 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f3078-05db-433e-8c17-244c238c4d2c service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:50.773372 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f3078-05db-433e-8c17-244c238c4d2c service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:50.773633 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f3078-05db-433e-8c17-244c238c4d2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:50.775212 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-876f3078-05db-433e-8c17-244c238c4d2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:50.860511 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f3078-05db-433e-8c17-244c238c4d2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:50.862075 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f3078-05db-433e-8c17-244c238c4d2c service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:50.880839 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 547/1094] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:52.037426 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c86b0-e762-402e-9de4-55cdd6656e15 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:52.038008 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c86b0-e762-402e-9de4-55cdd6656e15 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:52.038008 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c86b0-e762-402e-9de4-55cdd6656e15 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:52.039323 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 548/1095] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 14:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:52.045617 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350654-eae9-494f-8dc0-7e6ea1e249c1 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:52.046076 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350654-eae9-494f-8dc0-7e6ea1e249c1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:52.046200 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350654-eae9-494f-8dc0-7e6ea1e249c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:52.100713 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:52.122205 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a9350654-eae9-494f-8dc0-7e6ea1e249c1 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:52.123421 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a9350654-eae9-494f-8dc0-7e6ea1e249c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:52.219027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 548/1096] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 14:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:52.371386 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c86b0-e762-402e-9de4-55cdd6656e15 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:52.372907 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-046c86b0-e762-402e-9de4-55cdd6656e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:52.516048 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c86b0-e762-402e-9de4-55cdd6656e15 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:52.528607 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:52.631427 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272138ea-1626-4ad4-afa2-dd27200c29b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:52.631951 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272138ea-1626-4ad4-afa2-dd27200c29b4 service nova] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:52.632221 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272138ea-1626-4ad4-afa2-dd27200c29b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:52.632484 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272138ea-1626-4ad4-afa2-dd27200c29b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:52.634035 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-272138ea-1626-4ad4-afa2-dd27200c29b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:52.720228 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272138ea-1626-4ad4-afa2-dd27200c29b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:52.721846 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272138ea-1626-4ad4-afa2-dd27200c29b4 service nova] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:52.739159 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 549/1097] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:53.857760 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e909182-cc4e-40e0-9c32-9826691499cd None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:53.858210 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e909182-cc4e-40e0-9c32-9826691499cd None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:53.858304 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e909182-cc4e-40e0-9c32-9826691499cd None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:53.859639 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 549/1098] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 14:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:53.866044 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6e2e1-e48d-4561-be82-819c8e5d9af3 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:53.866454 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6e2e1-e48d-4561-be82-819c8e5d9af3 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:53.866561 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6e2e1-e48d-4561-be82-819c8e5d9af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:53.886138 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:53.942051 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-c1e6e2e1-e48d-4561-be82-819c8e5d9af3 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:53.943260 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-c1e6e2e1-e48d-4561-be82-819c8e5d9af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:54.039156 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 550/1099] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 14:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:54.192892 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e909182-cc4e-40e0-9c32-9826691499cd None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:54.194396 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-5e909182-cc4e-40e0-9c32-9826691499cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:54.305320 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:54.341559 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e909182-cc4e-40e0-9c32-9826691499cd None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:54.452099 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab242eb0-a9a4-4fe0-b013-4d736aa746d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:54.452622 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab242eb0-a9a4-4fe0-b013-4d736aa746d4 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:54.452892 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab242eb0-a9a4-4fe0-b013-4d736aa746d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:54.453157 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab242eb0-a9a4-4fe0-b013-4d736aa746d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:54.454712 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ab242eb0-a9a4-4fe0-b013-4d736aa746d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:54.530681 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:54.543766 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab242eb0-a9a4-4fe0-b013-4d736aa746d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:54.545378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab242eb0-a9a4-4fe0-b013-4d736aa746d4 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:54.563326 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 550/1100] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:55.685636 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e09f22-f958-4aac-b952-0da59dfc1330 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:55.686168 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e09f22-f958-4aac-b952-0da59dfc1330 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:55.686168 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e09f22-f958-4aac-b952-0da59dfc1330 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:55.687535 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 551/1101] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 14:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:55.694462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6539e7f-63a4-4967-a4f9-0f66b9e02b41 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:55.694896 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6539e7f-63a4-4967-a4f9-0f66b9e02b41 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:55.694999 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6539e7f-63a4-4967-a4f9-0f66b9e02b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:55.768744 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d6539e7f-63a4-4967-a4f9-0f66b9e02b41 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:55.770500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d6539e7f-63a4-4967-a4f9-0f66b9e02b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:55.865411 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 551/1102] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 14:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:56.018497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e09f22-f958-4aac-b952-0da59dfc1330 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:56.020055 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-76e09f22-f958-4aac-b952-0da59dfc1330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:56.105018 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:56.169538 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e09f22-f958-4aac-b952-0da59dfc1330 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:56.283053 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3304da1-a6ef-4b6c-80d8-09db84bd824c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:56.283684 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3304da1-a6ef-4b6c-80d8-09db84bd824c service nova] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:56.283953 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3304da1-a6ef-4b6c-80d8-09db84bd824c service nova] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:56.284212 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3304da1-a6ef-4b6c-80d8-09db84bd824c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:56.286316 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-a3304da1-a6ef-4b6c-80d8-09db84bd824c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:56.334098 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:56.374995 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3304da1-a6ef-4b6c-80d8-09db84bd824c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:56.376598 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3304da1-a6ef-4b6c-80d8-09db84bd824c service nova] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:56.394830 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 552/1103] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:57.444587 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674aa04-d21b-49d0-bff5-e4d46d27a525 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:57.445126 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674aa04-d21b-49d0-bff5-e4d46d27a525 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:57.445126 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674aa04-d21b-49d0-bff5-e4d46d27a525 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:57.446468 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 552/1104] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 14:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:57.452783 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b36ed-8dfc-430c-8de5-4bb800db043c None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:57.453159 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b36ed-8dfc-430c-8de5-4bb800db043c None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:57.453325 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b36ed-8dfc-430c-8de5-4bb800db043c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:57.528675 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-a59b36ed-8dfc-430c-8de5-4bb800db043c None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:57.529867 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-a59b36ed-8dfc-430c-8de5-4bb800db043c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:57.629723 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 553/1105] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 14:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:57.780216 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674aa04-d21b-49d0-bff5-e4d46d27a525 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:57.781722 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6674aa04-d21b-49d0-bff5-e4d46d27a525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:57.830732 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:57.926411 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674aa04-d21b-49d0-bff5-e4d46d27a525 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:58.038555 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d2380-6ac2-4669-9e19-1dbcd34a73a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:58.039078 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d2380-6ac2-4669-9e19-1dbcd34a73a5 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:58.039375 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d2380-6ac2-4669-9e19-1dbcd34a73a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:58.039635 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d2380-6ac2-4669-9e19-1dbcd34a73a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:58.041203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-654d2380-6ac2-4669-9e19-1dbcd34a73a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:58.047968 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:58.130636 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d2380-6ac2-4669-9e19-1dbcd34a73a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:58.132228 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d2380-6ac2-4669-9e19-1dbcd34a73a5 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:58.150602 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 553/1106] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:32:59.228120 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc1e21-5eea-48cc-b735-81ef2663b4bc None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:59.228699 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc1e21-5eea-48cc-b735-81ef2663b4bc None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:59.228699 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc1e21-5eea-48cc-b735-81ef2663b4bc None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:59.230013 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 554/1107] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 14:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:32:59.236386 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2235e4-9368-4f34-9144-77883e670990 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:59.236818 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2235e4-9368-4f34-9144-77883e670990 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:59.236992 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2235e4-9368-4f34-9144-77883e670990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:59.314325 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-0c2235e4-9368-4f34-9144-77883e670990 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:32:59.315763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0c2235e4-9368-4f34-9144-77883e670990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:59.382605 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:59.411816 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 554/1108] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 14:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:32:59.561314 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc1e21-5eea-48cc-b735-81ef2663b4bc None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:32:59.562725 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-1edc1e21-5eea-48cc-b735-81ef2663b4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:59.580833 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:59.708028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc1e21-5eea-48cc-b735-81ef2663b4bc None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:32:59.794119 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:32:59.821678 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c52b10-eb70-4c32-b38b-b4f04417195e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:32:59.822178 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c52b10-eb70-4c32-b38b-b4f04417195e service nova] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:32:59.822447 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c52b10-eb70-4c32-b38b-b4f04417195e service nova] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:32:59.822708 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c52b10-eb70-4c32-b38b-b4f04417195e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:32:59.824301 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-17c52b10-eb70-4c32-b38b-b4f04417195e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:32:59.912434 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c52b10-eb70-4c32-b38b-b4f04417195e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:32:59.914016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c52b10-eb70-4c32-b38b-b4f04417195e service nova] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:32:59.931999 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 555/1109] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:32:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:33:01.002537 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dd1c9-b468-4ef7-95e3-e47fc9a1449d None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:01.003075 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dd1c9-b468-4ef7-95e3-e47fc9a1449d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:01.003075 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dd1c9-b468-4ef7-95e3-e47fc9a1449d None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:01.004956 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 555/1110] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 14:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:33:01.011470 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a417c-bad6-44ec-8c95-7c089e47372a None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:01.011873 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a417c-bad6-44ec-8c95-7c089e47372a None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:01.012004 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a417c-bad6-44ec-8c95-7c089e47372a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:01.087825 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-df2a417c-bad6-44ec-8c95-7c089e47372a None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:33:01.089011 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-df2a417c-bad6-44ec-8c95-7c089e47372a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:01.130250 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:01.184879 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 556/1111] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 14:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:33:01.342627 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7dd1c9-b468-4ef7-95e3-e47fc9a1449d None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:33:01.344079 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2c7dd1c9-b468-4ef7-95e3-e47fc9a1449d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:01.491561 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7dd1c9-b468-4ef7-95e3-e47fc9a1449d None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:33:01.545409 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:01.607175 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f625a-e7b1-483b-82b4-5ec0350b9d71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:33:01.607720 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f625a-e7b1-483b-82b4-5ec0350b9d71 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:01.607993 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f625a-e7b1-483b-82b4-5ec0350b9d71 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:01.608259 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f625a-e7b1-483b-82b4-5ec0350b9d71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:01.609839 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f09f625a-e7b1-483b-82b4-5ec0350b9d71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:01.699119 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f625a-e7b1-483b-82b4-5ec0350b9d71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:33:01.700725 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f625a-e7b1-483b-82b4-5ec0350b9d71 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:33:01.718556 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 556/1112] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:33:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:33:02.774281 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98fa47-5be9-43ce-b1c6-c3ee426d6c25 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:02.774816 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98fa47-5be9-43ce-b1c6-c3ee426d6c25 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:02.774816 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98fa47-5be9-43ce-b1c6-c3ee426d6c25 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:02.776233 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 557/1113] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 14:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:33:02.782335 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5fd15-70c8-47bc-9f6f-113034288fdb None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:02.782807 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5fd15-70c8-47bc-9f6f-113034288fdb None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:02.783001 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5fd15-70c8-47bc-9f6f-113034288fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:02.857764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-2bd5fd15-70c8-47bc-9f6f-113034288fdb None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:33:02.858878 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2bd5fd15-70c8-47bc-9f6f-113034288fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:02.860867 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:02.954139 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 557/1114] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 14:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:33:03.104673 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98fa47-5be9-43ce-b1c6-c3ee426d6c25 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:33:03.106193 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-8f98fa47-5be9-43ce-b1c6-c3ee426d6c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:03.256956 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98fa47-5be9-43ce-b1c6-c3ee426d6c25 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:33:03.286641 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:03.371973 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9206bb-d4cb-49b6-aa44-ab76eae4b776 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:33:03.372573 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9206bb-d4cb-49b6-aa44-ab76eae4b776 service nova] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:03.372843 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9206bb-d4cb-49b6-aa44-ab76eae4b776 service nova] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:03.373107 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9206bb-d4cb-49b6-aa44-ab76eae4b776 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:03.374662 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-7f9206bb-d4cb-49b6-aa44-ab76eae4b776 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:03.462958 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9206bb-d4cb-49b6-aa44-ab76eae4b776 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:33:03.464565 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9206bb-d4cb-49b6-aa44-ab76eae4b776 service nova] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:33:03.481907 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 558/1115] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:33:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:33:05.564427 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b76d88-6d87-4dc5-99c7-b2dd0299fdff None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:05.565029 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b76d88-6d87-4dc5-99c7-b2dd0299fdff None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:05.565029 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b76d88-6d87-4dc5-99c7-b2dd0299fdff None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:05.566640 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 558/1116] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 14:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:33:05.572130 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a4f4c-be49-4918-9f69-8dc3ac2c80d6 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:05.572583 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a4f4c-be49-4918-9f69-8dc3ac2c80d6 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:05.572690 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a4f4c-be49-4918-9f69-8dc3ac2c80d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:05.614320 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:05.638923 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-5c4a4f4c-be49-4918-9f69-8dc3ac2c80d6 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:33:05.640130 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-5c4a4f4c-be49-4918-9f69-8dc3ac2c80d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:05.736838 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 559/1117] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 14:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:33:06.021670 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b76d88-6d87-4dc5-99c7-b2dd0299fdff None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:33:06.023250 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a1b76d88-6d87-4dc5-99c7-b2dd0299fdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:06.126579 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b76d88-6d87-4dc5-99c7-b2dd0299fdff None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:33:06.158535 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:06.205171 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad93f5b-b8e1-4a1b-b6b6-e92350f973f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:33:06.205816 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad93f5b-b8e1-4a1b-b6b6-e92350f973f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:06.206090 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad93f5b-b8e1-4a1b-b6b6-e92350f973f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:06.206354 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad93f5b-b8e1-4a1b-b6b6-e92350f973f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:06.207973 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-fad93f5b-b8e1-4a1b-b6b6-e92350f973f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:06.314730 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad93f5b-b8e1-4a1b-b6b6-e92350f973f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:33:06.316408 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad93f5b-b8e1-4a1b-b6b6-e92350f973f3 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:33:06.334286 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 559/1118] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:33:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 14:33:08.655726 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cd478-0dba-4db4-9390-c4b1b7f0be07 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:08.656298 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cd478-0dba-4db4-9390-c4b1b7f0be07 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:08.656298 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cd478-0dba-4db4-9390-c4b1b7f0be07 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:08.657773 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 560/1119] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 14:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:33:08.663645 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a55b8-7192-4d26-af8b-2a07230fa96e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:08.664081 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a55b8-7192-4d26-af8b-2a07230fa96e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:08.664218 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a55b8-7192-4d26-af8b-2a07230fa96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:08.732858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-1d0a55b8-7192-4d26-af8b-2a07230fa96e None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:33:08.734064 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-1d0a55b8-7192-4d26-af8b-2a07230fa96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:08.746806 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:08.831527 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 560/1120] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 14:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:33:09.116834 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cd478-0dba-4db4-9390-c4b1b7f0be07 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:33:09.118281 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-a54cd478-0dba-4db4-9390-c4b1b7f0be07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:09.219438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cd478-0dba-4db4-9390-c4b1b7f0be07 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:33:09.240002 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:09.307741 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cef68-896f-43b7-8f95-ff3462079b6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:33:09.308348 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cef68-896f-43b7-8f95-ff3462079b6b service neutron] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:09.308620 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cef68-896f-43b7-8f95-ff3462079b6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:09.308893 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cef68-896f-43b7-8f95-ff3462079b6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:09.310451 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-321cef68-896f-43b7-8f95-ff3462079b6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:09.419998 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cef68-896f-43b7-8f95-ff3462079b6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:33:09.421428 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cef68-896f-43b7-8f95-ff3462079b6b service neutron] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:33:09.439302 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 561/1121] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:33:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:33:11.907898 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4627c75-f5b9-405e-b319-bf332146c7cd None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:11.908932 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4627c75-f5b9-405e-b319-bf332146c7cd None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:11.909084 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4627c75-f5b9-405e-b319-bf332146c7cd None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:11.910430 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 561/1122] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 14:33:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:33:11.916083 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ba33c-3a8d-4a3b-9ae2-8dd54d84eda6 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:11.916466 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ba33c-3a8d-4a3b-9ae2-8dd54d84eda6 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:11.916622 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ba33c-3a8d-4a3b-9ae2-8dd54d84eda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:11.964241 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:11.982160 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-470ba33c-3a8d-4a3b-9ae2-8dd54d84eda6 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:33:11.983396 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-470ba33c-3a8d-4a3b-9ae2-8dd54d84eda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:12.079104 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 562/1123] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 14:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:33:12.370608 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4627c75-f5b9-405e-b319-bf332146c7cd None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:33:12.372174 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c4627c75-f5b9-405e-b319-bf332146c7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:12.474292 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4627c75-f5b9-405e-b319-bf332146c7cd None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:33:12.514508 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:12.547932 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe1b2e-d787-4427-95c9-68c94d579320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:33:12.548536 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe1b2e-d787-4427-95c9-68c94d579320 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:12.549439 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe1b2e-d787-4427-95c9-68c94d579320 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:12.549713 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe1b2e-d787-4427-95c9-68c94d579320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:12.551312 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-bbbe1b2e-d787-4427-95c9-68c94d579320 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:12.658824 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe1b2e-d787-4427-95c9-68c94d579320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:33:12.660284 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe1b2e-d787-4427-95c9-68c94d579320 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:33:12.677620 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 562/1124] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:33:14.961811 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab310cd-239b-446b-9fe9-6309f276edcb None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:14.962405 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab310cd-239b-446b-9fe9-6309f276edcb None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:14.962405 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab310cd-239b-446b-9fe9-6309f276edcb None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:14.963727 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 563/1125] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 14:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:33:14.969118 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ca030-2a70-42de-a704-8dc007d4764c None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:14.969542 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ca030-2a70-42de-a704-8dc007d4764c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:14.969644 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ca030-2a70-42de-a704-8dc007d4764c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:15.035867 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-353ca030-2a70-42de-a704-8dc007d4764c None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:33:15.037044 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-353ca030-2a70-42de-a704-8dc007d4764c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:15.057900 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:15.134428 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 563/1126] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 14:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 14:33:15.420402 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab310cd-239b-446b-9fe9-6309f276edcb None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:33:15.421912 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-aab310cd-239b-446b-9fe9-6309f276edcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:15.524839 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab310cd-239b-446b-9fe9-6309f276edcb None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:33:15.544861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:15.601901 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ffa55-f9ce-4974-9cb4-ce307a100da8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:33:15.602532 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ffa55-f9ce-4974-9cb4-ce307a100da8 service neutron] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:15.602806 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ffa55-f9ce-4974-9cb4-ce307a100da8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:15.603066 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ffa55-f9ce-4974-9cb4-ce307a100da8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:15.604711 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-5d1ffa55-f9ce-4974-9cb4-ce307a100da8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:15.714374 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ffa55-f9ce-4974-9cb4-ce307a100da8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:33:15.715768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ffa55-f9ce-4974-9cb4-ce307a100da8 service neutron] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:33:15.733786 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 564/1127] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:33:18.025756 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a22b12-cbe3-4304-a4af-f646ddcc655c None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:18.026312 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a22b12-cbe3-4304-a4af-f646ddcc655c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:18.026312 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a22b12-cbe3-4304-a4af-f646ddcc655c None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:18.027664 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 564/1128] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 14:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:33:18.033923 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae301e-37c8-4491-95e8-3399a431b225 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:18.034354 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae301e-37c8-4491-95e8-3399a431b225 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:18.034451 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae301e-37c8-4491-95e8-3399a431b225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:18.097163 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:18.099945 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-38ae301e-37c8-4491-95e8-3399a431b225 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:33:18.101624 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-38ae301e-37c8-4491-95e8-3399a431b225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:18.196593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 565/1129] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 14:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:33:18.483560 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a22b12-cbe3-4304-a4af-f646ddcc655c None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:33:18.485065 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-76a22b12-cbe3-4304-a4af-f646ddcc655c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:18.587563 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a22b12-cbe3-4304-a4af-f646ddcc655c None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:33:18.635916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:18.661412 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba3b08-4a47-4a50-a28f-dc5e39d9d659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:33:18.662018 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba3b08-4a47-4a50-a28f-dc5e39d9d659 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:18.662289 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba3b08-4a47-4a50-a28f-dc5e39d9d659 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:18.662555 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba3b08-4a47-4a50-a28f-dc5e39d9d659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:18.664198 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d1ba3b08-4a47-4a50-a28f-dc5e39d9d659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:18.775012 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba3b08-4a47-4a50-a28f-dc5e39d9d659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:33:18.776386 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba3b08-4a47-4a50-a28f-dc5e39d9d659 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:33:18.794464 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 565/1130] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:33:21.017659 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f1e4e-712c-4a8a-a42c-6172ae6cd22e None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:21.018235 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f1e4e-712c-4a8a-a42c-6172ae6cd22e None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:21.018235 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f1e4e-712c-4a8a-a42c-6172ae6cd22e None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:21.019610 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 566/1131] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 14:33:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:33:21.024901 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba88d3-114e-4d3c-ad05-04e7024e26cb None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:21.025296 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba88d3-114e-4d3c-ad05-04e7024e26cb None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:21.025441 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba88d3-114e-4d3c-ad05-04e7024e26cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:21.098248 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d4ba88d3-114e-4d3c-ad05-04e7024e26cb None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:33:21.099460 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d4ba88d3-114e-4d3c-ad05-04e7024e26cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:21.130126 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:21.200201 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 566/1132] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 14:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:33:21.502823 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f1e4e-712c-4a8a-a42c-6172ae6cd22e None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:33:21.504454 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-ca6f1e4e-712c-4a8a-a42c-6172ae6cd22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:21.619316 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f1e4e-712c-4a8a-a42c-6172ae6cd22e None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:33:21.648893 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:21.696252 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc10723-4b8f-4771-8d90-5b77025e1b14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:33:21.696899 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc10723-4b8f-4771-8d90-5b77025e1b14 service neutron] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:21.697217 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc10723-4b8f-4771-8d90-5b77025e1b14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:21.697489 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc10723-4b8f-4771-8d90-5b77025e1b14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:21.699103 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-0dc10723-4b8f-4771-8d90-5b77025e1b14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:21.818579 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc10723-4b8f-4771-8d90-5b77025e1b14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:33:21.819945 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc10723-4b8f-4771-8d90-5b77025e1b14 service neutron] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:33:21.839365 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 567/1133] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:33:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:33:24.069151 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28026377-1577-4a52-b0e5-a783e197899a None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:24.069685 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28026377-1577-4a52-b0e5-a783e197899a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:24.069685 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28026377-1577-4a52-b0e5-a783e197899a None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:24.071027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 567/1134] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 14:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:33:24.076366 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eea65b-51d8-4678-8fc8-e33972daa948 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:24.076794 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eea65b-51d8-4678-8fc8-e33972daa948 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:24.076889 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eea65b-51d8-4678-8fc8-e33972daa948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:24.142232 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-69eea65b-51d8-4678-8fc8-e33972daa948 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:33:24.143363 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-69eea65b-51d8-4678-8fc8-e33972daa948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:24.151569 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:24.240194 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 568/1135] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 14:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:33:24.523924 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28026377-1577-4a52-b0e5-a783e197899a None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:33:24.525619 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-28026377-1577-4a52-b0e5-a783e197899a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:24.628083 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28026377-1577-4a52-b0e5-a783e197899a None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:33:24.684780 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:24.704245 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742363c-9a6f-43c3-b35b-63c4142cd097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:33:24.705025 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742363c-9a6f-43c3-b35b-63c4142cd097 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:24.705298 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742363c-9a6f-43c3-b35b-63c4142cd097 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:24.705562 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742363c-9a6f-43c3-b35b-63c4142cd097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:24.707137 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-1742363c-9a6f-43c3-b35b-63c4142cd097 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:24.813939 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742363c-9a6f-43c3-b35b-63c4142cd097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:33:24.815323 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742363c-9a6f-43c3-b35b-63c4142cd097 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:33:24.833385 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 568/1136] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:33:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:33:27.024111 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a57bf-b1bc-4438-9103-8f895a7a55fd None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:27.024677 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a57bf-b1bc-4438-9103-8f895a7a55fd None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:27.024677 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a57bf-b1bc-4438-9103-8f895a7a55fd None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:27.025985 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 569/1137] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 14:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:33:27.031781 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cbc8e-a3f5-4c8b-ae75-865f33588577 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:27.032288 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cbc8e-a3f5-4c8b-ae75-865f33588577 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:27.032442 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cbc8e-a3f5-4c8b-ae75-865f33588577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:27.097001 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9f2cbc8e-a3f5-4c8b-ae75-865f33588577 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:33:27.098184 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9f2cbc8e-a3f5-4c8b-ae75-865f33588577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:27.136519 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:27.194536 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 569/1138] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 14:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:33:27.479182 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a57bf-b1bc-4438-9103-8f895a7a55fd None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:33:27.480728 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-875a57bf-b1bc-4438-9103-8f895a7a55fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:27.583854 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a57bf-b1bc-4438-9103-8f895a7a55fd None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:33:27.621444 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:27.658870 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e394a-6d22-4198-82ee-6ad34d50513d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:33:27.659502 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e394a-6d22-4198-82ee-6ad34d50513d service neutron] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:27.659779 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e394a-6d22-4198-82ee-6ad34d50513d service neutron] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:27.660046 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e394a-6d22-4198-82ee-6ad34d50513d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:27.661621 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-4a3e394a-6d22-4198-82ee-6ad34d50513d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:27.769940 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e394a-6d22-4198-82ee-6ad34d50513d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:33:27.771316 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e394a-6d22-4198-82ee-6ad34d50513d service neutron] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:33:27.789183 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 570/1139] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:33:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:33:29.956656 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119129f-5199-47ec-b632-b236abf6b5d6 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:29.957254 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119129f-5199-47ec-b632-b236abf6b5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:29.957254 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119129f-5199-47ec-b632-b236abf6b5d6 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:29.958526 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 570/1140] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 14:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:33:29.963892 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61b282-6ece-43e9-a7cf-38c2b7dcd7c2 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:29.964330 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61b282-6ece-43e9-a7cf-38c2b7dcd7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:29.964446 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61b282-6ece-43e9-a7cf-38c2b7dcd7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:30.029584 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-dd61b282-6ece-43e9-a7cf-38c2b7dcd7c2 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:33:30.031506 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-dd61b282-6ece-43e9-a7cf-38c2b7dcd7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:30.050046 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:30.127768 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 571/1141] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 14:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:33:30.428013 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a169d9e-4d14-40d6-9dcb-5544d0beddc9 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:30.428322 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a169d9e-4d14-40d6-9dcb-5544d0beddc9 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:30.428593 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a169d9e-4d14-40d6-9dcb-5544d0beddc9 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:30.429916 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 571/1142] 10.222.0.16 () {54 vars in 964 bytes} [Wed Oct 12 14:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:33:30.435685 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff9cb0-235e-491c-8724-3b238e6cbb1c None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:30.435963 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff9cb0-235e-491c-8724-3b238e6cbb1c None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:30.436234 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff9cb0-235e-491c-8724-3b238e6cbb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:30.511332 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-faff9cb0-235e-491c-8724-3b238e6cbb1c None None] MFA Rules not processed for user `9d0d533e36ff4a5186f8704480b58646`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:33:30.512517 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-faff9cb0-235e-491c-8724-3b238e6cbb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:30.588664 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 572/1143] 10.222.0.16 () {58 vars in 1061 bytes} [Wed Oct 12 14:33:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:33:30.598520 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a169d9e-4d14-40d6-9dcb-5544d0beddc9 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:33:30.600036 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6a169d9e-4d14-40d6-9dcb-5544d0beddc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:30.702378 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a169d9e-4d14-40d6-9dcb-5544d0beddc9 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:33:30.769595 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:30.776656 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba019129-022a-4fc7-873d-ef4471bf261b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:33:30.777257 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba019129-022a-4fc7-873d-ef4471bf261b service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:30.777528 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba019129-022a-4fc7-873d-ef4471bf261b service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:30.777793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba019129-022a-4fc7-873d-ef4471bf261b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:30.779398 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ba019129-022a-4fc7-873d-ef4471bf261b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:30.886530 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba019129-022a-4fc7-873d-ef4471bf261b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:33:30.888004 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba019129-022a-4fc7-873d-ef4471bf261b service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:33:30.906305 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 572/1144] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:33:33.512113 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c542ae1-3ded-4092-9534-52f7efff0dd8 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:33.512691 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c542ae1-3ded-4092-9534-52f7efff0dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:33.512691 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c542ae1-3ded-4092-9534-52f7efff0dd8 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:33.514165 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 573/1145] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 14:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:33:33.519930 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f6e22-663e-455d-a31a-7db7cc2a5bf5 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:33.520370 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f6e22-663e-455d-a31a-7db7cc2a5bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:33.520468 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f6e22-663e-455d-a31a-7db7cc2a5bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:33.588185 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-e41f6e22-663e-455d-a31a-7db7cc2a5bf5 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:33:33.589377 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-e41f6e22-663e-455d-a31a-7db7cc2a5bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:33.636995 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:33.685304 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 573/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 14:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:33:33.970029 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c542ae1-3ded-4092-9534-52f7efff0dd8 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:33:33.971590 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-6c542ae1-3ded-4092-9534-52f7efff0dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:33.987186 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:34.076385 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c542ae1-3ded-4092-9534-52f7efff0dd8 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:33:34.150531 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917d99c-c295-4bbb-b794-8537b78737da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:33:34.151127 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917d99c-c295-4bbb-b794-8537b78737da service neutron] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:34.151427 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917d99c-c295-4bbb-b794-8537b78737da service neutron] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:34.151691 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917d99c-c295-4bbb-b794-8537b78737da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:34.153253 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-e917d99c-c295-4bbb-b794-8537b78737da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:34.219876 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:34.265209 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917d99c-c295-4bbb-b794-8537b78737da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:33:34.266665 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917d99c-c295-4bbb-b794-8537b78737da service neutron] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:33:34.284364 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 574/1147] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:33:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 14:33:43.683780 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=22 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:44.282008 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=25 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:54.305092 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7019358-9a88-4481-9d04-e97c6bdfce8a None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:54.305655 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7019358-9a88-4481-9d04-e97c6bdfce8a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:54.305655 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7019358-9a88-4481-9d04-e97c6bdfce8a None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:54.306972 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 574/1148] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:33:54.312901 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748dda9-99a6-4347-a10c-fd87c3b336c1 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:54.313297 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748dda9-99a6-4347-a10c-fd87c3b336c1 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:54.313438 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748dda9-99a6-4347-a10c-fd87c3b336c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:54.377780 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-c748dda9-99a6-4347-a10c-fd87c3b336c1 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:33:54.378965 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-c748dda9-99a6-4347-a10c-fd87c3b336c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:54.473625 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 575/1149] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:33:54.771019 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a7257-b65f-4df0-8b11-2314823b9576 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:54.771331 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a7257-b65f-4df0-8b11-2314823b9576 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:54.771608 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a7257-b65f-4df0-8b11-2314823b9576 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:54.772926 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 575/1150] 10.222.0.16 () {54 vars in 962 bytes} [Wed Oct 12 14:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:33:54.778436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36715880-33a3-4e14-83f1-2dcd080f0d7b None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:54.778712 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36715880-33a3-4e14-83f1-2dcd080f0d7b None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:54.778979 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36715880-33a3-4e14-83f1-2dcd080f0d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:54.852812 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-36715880-33a3-4e14-83f1-2dcd080f0d7b None None] MFA Rules not processed for user `2b95d3ba7df041aab3281129f6516d20`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:33:54.853918 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-36715880-33a3-4e14-83f1-2dcd080f0d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:54.874795 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:54.930528 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 576/1151] 10.222.0.16 () {58 vars in 1059 bytes} [Wed Oct 12 14:33:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:33:54.938455 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a7257-b65f-4df0-8b11-2314823b9576 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:33:54.939997 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-b94a7257-b65f-4df0-8b11-2314823b9576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:55.041743 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a7257-b65f-4df0-8b11-2314823b9576 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:33:55.115479 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3f756-a2c1-4f5a-ad0f-f3463a78c655 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b95d3ba7df041aab3281129f6516d20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:33:55.116006 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3f756-a2c1-4f5a-ad0f-f3463a78c655 service glance] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:55.116283 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3f756-a2c1-4f5a-ad0f-f3463a78c655 service glance] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:55.116550 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3f756-a2c1-4f5a-ad0f-f3463a78c655 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:55.118141 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7ab3f756-a2c1-4f5a-ad0f-f3463a78c655 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:55.171920 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:55.225701 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3f756-a2c1-4f5a-ad0f-f3463a78c655 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:33:55.227129 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3f756-a2c1-4f5a-ad0f-f3463a78c655 service glance] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:33:55.245966 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 576/1152] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:33:57.480927 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a6235-ad71-4e04-a9c5-9e0f615d396b None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:57.481497 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a6235-ad71-4e04-a9c5-9e0f615d396b None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:57.481611 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a6235-ad71-4e04-a9c5-9e0f615d396b None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:57.482883 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 577/1153] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:33:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:33:57.488247 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52bd99-2d6e-4cce-8d2b-2424ddb381b8 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:57.488670 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52bd99-2d6e-4cce-8d2b-2424ddb381b8 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:57.488771 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52bd99-2d6e-4cce-8d2b-2424ddb381b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:57.554183 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-fe52bd99-2d6e-4cce-8d2b-2424ddb381b8 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:33:57.555434 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-fe52bd99-2d6e-4cce-8d2b-2424ddb381b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:57.654224 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 577/1154] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:33:57.927020 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a6235-ad71-4e04-a9c5-9e0f615d396b None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:33:57.928571 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-5c0a6235-ad71-4e04-a9c5-9e0f615d396b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:33:58.053492 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a6235-ad71-4e04-a9c5-9e0f615d396b None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:33:58.102408 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:33:58.146157 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fb156-bb74-4580-82aa-7c1f314390a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:33:58.146707 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fb156-bb74-4580-82aa-7c1f314390a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:33:58.146982 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fb156-bb74-4580-82aa-7c1f314390a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:33:58.147286 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fb156-bb74-4580-82aa-7c1f314390a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:33:58.147914 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2fb156-bb74-4580-82aa-7c1f314390a3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:33:58.149020 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: WARNING py.warnings [None req-3a2fb156-bb74-4580-82aa-7c1f314390a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 14:33:58.149020 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: warnings.warn(msg) Oct 12 14:33:58.149020 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: Oct 12 14:33:58.150162 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2fb156-bb74-4580-82aa-7c1f314390a3 admin admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:33:58.169470 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 578/1155] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Oct 12 14:33:57 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 14:34:00.340682 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b74bf8-9d6d-42f0-9313-210408af4db7 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:34:00.341331 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b74bf8-9d6d-42f0-9313-210408af4db7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:34:00.341331 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b74bf8-9d6d-42f0-9313-210408af4db7 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:34:00.342533 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 578/1156] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:34:00.348628 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253038c3-f270-43d7-a33d-dd3b10740553 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:34:00.349038 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253038c3-f270-43d7-a33d-dd3b10740553 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:34:00.349191 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253038c3-f270-43d7-a33d-dd3b10740553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:34:00.414381 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-253038c3-f270-43d7-a33d-dd3b10740553 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:34:00.415600 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-253038c3-f270-43d7-a33d-dd3b10740553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:34:00.509468 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 579/1157] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:34:00.804324 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b74bf8-9d6d-42f0-9313-210408af4db7 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:34:00.806020 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-27b74bf8-9d6d-42f0-9313-210408af4db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:34:00.817095 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:34:00.953868 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b74bf8-9d6d-42f0-9313-210408af4db7 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:34:01.032690 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:34:01.066213 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77b06d-821f-4354-ac4b-270b3e6c858d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0aeed5f32a1477dbd9d3e934669705b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:34:01.066737 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77b06d-821f-4354-ac4b-270b3e6c858d service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:34:01.067006 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77b06d-821f-4354-ac4b-270b3e6c858d service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:34:01.067304 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77b06d-821f-4354-ac4b-270b3e6c858d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:34:01.068858 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8c77b06d-821f-4354-ac4b-270b3e6c858d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:34:01.156270 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77b06d-821f-4354-ac4b-270b3e6c858d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:34:01.157832 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77b06d-821f-4354-ac4b-270b3e6c858d service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:34:01.175597 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 579/1158] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:34:03.496416 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef3140-ba2e-4456-89c3-91015c987da0 None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:34:03.496979 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef3140-ba2e-4456-89c3-91015c987da0 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:34:03.496979 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef3140-ba2e-4456-89c3-91015c987da0 None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:34:03.498275 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 580/1159] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:34:03.503627 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e985f-81ff-46f6-9329-43288ff575cf None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:34:03.504030 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e985f-81ff-46f6-9329-43288ff575cf None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:34:03.504261 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e985f-81ff-46f6-9329-43288ff575cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:34:03.570413 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-036e985f-81ff-46f6-9329-43288ff575cf None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:34:03.571649 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-036e985f-81ff-46f6-9329-43288ff575cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:34:03.623764 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:34:03.674570 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 580/1160] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:34:03.962283 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef3140-ba2e-4456-89c3-91015c987da0 None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:34:03.964377 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-b0ef3140-ba2e-4456-89c3-91015c987da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:34:03.988244 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:34:04.066471 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef3140-ba2e-4456-89c3-91015c987da0 None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:34:04.140460 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cadf0c-8035-4de9-ad3a-fae50a8eadbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:34:04.140976 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cadf0c-8035-4de9-ad3a-fae50a8eadbf service neutron] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:34:04.141244 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cadf0c-8035-4de9-ad3a-fae50a8eadbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:34:04.141506 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cadf0c-8035-4de9-ad3a-fae50a8eadbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:34:04.143069 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-a0cadf0c-8035-4de9-ad3a-fae50a8eadbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:34:04.216686 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:34:04.251371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cadf0c-8035-4de9-ad3a-fae50a8eadbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:34:04.252793 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cadf0c-8035-4de9-ad3a-fae50a8eadbf service neutron] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:34:04.271374 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 581/1161] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:34:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:34:06.468228 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe6c69-32d4-44fe-b024-a375a47d6a3c None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:34:06.468781 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe6c69-32d4-44fe-b024-a375a47d6a3c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:34:06.468781 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe6c69-32d4-44fe-b024-a375a47d6a3c None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:34:06.470018 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 581/1162] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:34:06.475299 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3dc91-c898-47f7-a402-1aa409bdcc3f None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:34:06.475726 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3dc91-c898-47f7-a402-1aa409bdcc3f None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:34:06.475830 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3dc91-c898-47f7-a402-1aa409bdcc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:34:06.540411 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-18a3dc91-c898-47f7-a402-1aa409bdcc3f None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:34:06.542105 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-18a3dc91-c898-47f7-a402-1aa409bdcc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:34:06.638510 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 582/1163] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:34:06.930883 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe6c69-32d4-44fe-b024-a375a47d6a3c None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:34:06.933131 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-e7fe6c69-32d4-44fe-b024-a375a47d6a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:34:07.034475 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe6c69-32d4-44fe-b024-a375a47d6a3c None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:34:07.107202 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:34:07.108688 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2521e4-941f-4a1c-975b-07c29e371e72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:34:07.109317 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2521e4-941f-4a1c-975b-07c29e371e72 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:34:07.109596 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2521e4-941f-4a1c-975b-07c29e371e72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:34:07.110054 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2521e4-941f-4a1c-975b-07c29e371e72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:34:07.112136 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-1b2521e4-941f-4a1c-975b-07c29e371e72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:34:07.218474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2521e4-941f-4a1c-975b-07c29e371e72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:34:07.219851 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2521e4-941f-4a1c-975b-07c29e371e72 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:34:07.237846 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 582/1164] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:34:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:34:09.571684 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c151b8c-6c18-4c6c-a9ae-28c8344e252c None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:34:09.572277 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c151b8c-6c18-4c6c-a9ae-28c8344e252c None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:34:09.572277 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c151b8c-6c18-4c6c-a9ae-28c8344e252c None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:34:09.574135 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 583/1165] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:34:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:34:09.579576 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fcedb-1ab3-4a9a-a0de-cfb28f4a13e2 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:34:09.580010 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fcedb-1ab3-4a9a-a0de-cfb28f4a13e2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:34:09.580121 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fcedb-1ab3-4a9a-a0de-cfb28f4a13e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:34:09.647319 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-f22fcedb-1ab3-4a9a-a0de-cfb28f4a13e2 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:34:09.648522 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f22fcedb-1ab3-4a9a-a0de-cfb28f4a13e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:34:09.701740 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:34:09.744982 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 583/1166] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:34:10.034442 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c151b8c-6c18-4c6c-a9ae-28c8344e252c None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:34:10.036011 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-4c151b8c-6c18-4c6c-a9ae-28c8344e252c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:34:10.073817 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:34:10.140219 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c151b8c-6c18-4c6c-a9ae-28c8344e252c None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:34:10.214258 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609a9d7-9934-4ba6-9d7f-472beb33fd8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:34:10.214861 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609a9d7-9934-4ba6-9d7f-472beb33fd8d service neutron] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:34:10.215131 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609a9d7-9934-4ba6-9d7f-472beb33fd8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:34:10.215421 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609a9d7-9934-4ba6-9d7f-472beb33fd8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:34:10.217002 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-2609a9d7-9934-4ba6-9d7f-472beb33fd8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:34:10.296860 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:34:10.325599 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609a9d7-9934-4ba6-9d7f-472beb33fd8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:34:10.327016 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609a9d7-9934-4ba6-9d7f-472beb33fd8d service neutron] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:34:10.346049 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 584/1167] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:34:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:34:12.746176 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f46c4d-8c84-4a92-90cb-b8f5880006ca None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:34:12.746745 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f46c4d-8c84-4a92-90cb-b8f5880006ca None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:34:12.746745 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f46c4d-8c84-4a92-90cb-b8f5880006ca None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:34:12.748082 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 584/1168] 10.222.0.16 () {54 vars in 927 bytes} [Wed Oct 12 14:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:34:12.753309 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558faed-37aa-4208-8ced-0c78244141dd None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:34:12.753733 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558faed-37aa-4208-8ced-0c78244141dd None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:34:12.753850 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558faed-37aa-4208-8ced-0c78244141dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:34:12.819841 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-d558faed-37aa-4208-8ced-0c78244141dd None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:34:12.821031 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-d558faed-37aa-4208-8ced-0c78244141dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:34:12.915666 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 585/1169] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Oct 12 14:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:34:13.204773 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f46c4d-8c84-4a92-90cb-b8f5880006ca None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:34:13.206295 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-e0f46c4d-8c84-4a92-90cb-b8f5880006ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:34:13.312115 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f46c4d-8c84-4a92-90cb-b8f5880006ca None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:34:13.387167 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d981f6-e978-487c-97d6-980b7720eaaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:34:13.387784 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d981f6-e978-487c-97d6-980b7720eaaa service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:34:13.388061 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d981f6-e978-487c-97d6-980b7720eaaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:34:13.388330 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d981f6-e978-487c-97d6-980b7720eaaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:34:13.389911 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-37d981f6-e978-487c-97d6-980b7720eaaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:34:13.398297 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:34:13.497104 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d981f6-e978-487c-97d6-980b7720eaaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:34:13.498443 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d981f6-e978-487c-97d6-980b7720eaaa service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:34:13.516462 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 585/1170] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:34:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:34:22.913973 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=82 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:34:23.514276 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=50 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:35:26.849045 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27530ddd-ed03-4ccc-a017-ba83faa3405e None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:26.849593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27530ddd-ed03-4ccc-a017-ba83faa3405e None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:26.849593 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27530ddd-ed03-4ccc-a017-ba83faa3405e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:26.923687 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-27530ddd-ed03-4ccc-a017-ba83faa3405e None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:35:26.924866 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-27530ddd-ed03-4ccc-a017-ba83faa3405e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:35:27.019435 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 586/1171] 10.222.0.16 () {58 vars in 958 bytes} [Wed Oct 12 14:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:35:27.027574 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed5105-fc28-4b2a-a1a2-434c83d83f35 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:27.028059 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed5105-fc28-4b2a-a1a2-434c83d83f35 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:27.028162 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed5105-fc28-4b2a-a1a2-434c83d83f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:27.102424 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-98ed5105-fc28-4b2a-a1a2-434c83d83f35 None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:35:27.103562 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-98ed5105-fc28-4b2a-a1a2-434c83d83f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:35:27.197216 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 586/1172] 10.222.0.16 () {58 vars in 958 bytes} [Wed Oct 12 14:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:35:27.201981 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27530ddd-ed03-4ccc-a017-ba83faa3405e None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:35:27.203766 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-27530ddd-ed03-4ccc-a017-ba83faa3405e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:35:27.214804 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27530ddd-ed03-4ccc-a017-ba83faa3405e None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:35:27.275342 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:35:27.308253 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f358f15-129e-4d40-9344-7afd1c372aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:35:27.308853 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f358f15-129e-4d40-9344-7afd1c372aea admin admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:27.309126 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f358f15-129e-4d40-9344-7afd1c372aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:27.309393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f358f15-129e-4d40-9344-7afd1c372aea admin admin] PATH_INFO: `/v3/domains` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:27.310024 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f358f15-129e-4d40-9344-7afd1c372aea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 14:35:27.310318 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f358f15-129e-4d40-9344-7afd1c372aea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:35:27.311345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: WARNING py.warnings [None req-3f358f15-129e-4d40-9344-7afd1c372aea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 12 14:35:27.311345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: warnings.warn(msg) Oct 12 14:35:27.311345 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: Oct 12 14:35:27.312894 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f358f15-129e-4d40-9344-7afd1c372aea admin admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:35:27.323720 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 587/1173] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Oct 12 14:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 14:35:27.333707 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed5105-fc28-4b2a-a1a2-434c83d83f35 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:35:27.342658 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed5105-fc28-4b2a-a1a2-434c83d83f35 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:35:27.403027 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:35:27.436725 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343f8ae-1b32-4cfd-9531-c513a426befc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:35:27.437438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343f8ae-1b32-4cfd-9531-c513a426befc admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:27.437710 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343f8ae-1b32-4cfd-9531-c513a426befc admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:27.437978 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343f8ae-1b32-4cfd-9531-c513a426befc admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:27.438606 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343f8ae-1b32-4cfd-9531-c513a426befc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:35:27.439646 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: WARNING py.warnings [None req-8343f8ae-1b32-4cfd-9531-c513a426befc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 14:35:27.439646 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: warnings.warn(msg) Oct 12 14:35:27.439646 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: Oct 12 14:35:27.440823 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343f8ae-1b32-4cfd-9531-c513a426befc admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:35:27.495594 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 587/1174] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Oct 12 14:35:27 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:35:27.500090 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f358f15-129e-4d40-9344-7afd1c372aea admin admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:35:27.502061 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-3f358f15-129e-4d40-9344-7afd1c372aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:35:27.535878 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f358f15-129e-4d40-9344-7afd1c372aea admin admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:35:27.629855 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09650f9f-849b-479f-b1e8-2e5384430d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:35:27.630488 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09650f9f-849b-479f-b1e8-2e5384430d67 admin admin] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:27.630761 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09650f9f-849b-479f-b1e8-2e5384430d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:27.631028 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09650f9f-849b-479f-b1e8-2e5384430d67 admin admin] PATH_INFO: `/v3/users` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:27.631689 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09650f9f-849b-479f-b1e8-2e5384430d67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:35:27.632751 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: WARNING py.warnings [None req-09650f9f-849b-479f-b1e8-2e5384430d67 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 14:35:27.632751 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: warnings.warn(msg) Oct 12 14:35:27.632751 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: Oct 12 14:35:27.633945 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09650f9f-849b-479f-b1e8-2e5384430d67 admin admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:35:27.675864 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 588/1175] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Oct 12 14:35:27 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 14:35:27.679903 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343f8ae-1b32-4cfd-9531-c513a426befc admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:35:27.681442 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8343f8ae-1b32-4cfd-9531-c513a426befc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:35:27.743059 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343f8ae-1b32-4cfd-9531-c513a426befc admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:35:27.825755 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:35:27.836655 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2294e-c533-4cf3-b4bb-5b4afbdb8d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:35:27.837260 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2294e-c533-4cf3-b4bb-5b4afbdb8d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:27.837534 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2294e-c533-4cf3-b4bb-5b4afbdb8d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:27.837802 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2294e-c533-4cf3-b4bb-5b4afbdb8d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:27.838438 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2294e-c533-4cf3-b4bb-5b4afbdb8d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:35:27.840062 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2294e-c533-4cf3-b4bb-5b4afbdb8d4a admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:35:27.852376 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 588/1176] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Oct 12 14:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 14:35:27.856410 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09650f9f-849b-479f-b1e8-2e5384430d67 admin admin] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:35:27.857917 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-09650f9f-849b-479f-b1e8-2e5384430d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:35:27.869637 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09650f9f-849b-479f-b1e8-2e5384430d67 admin admin] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:35:27.894844 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:35:27.965585 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eca5a4-40a4-4366-b89e-5f373ca5e70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:35:27.966089 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eca5a4-40a4-4366-b89e-5f373ca5e70f admin admin] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:27.966359 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eca5a4-40a4-4366-b89e-5f373ca5e70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:27.966618 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eca5a4-40a4-4366-b89e-5f373ca5e70f admin admin] PATH_INFO: `/v3/roles` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:27.967276 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eca5a4-40a4-4366-b89e-5f373ca5e70f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:35:27.968936 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: WARNING py.warnings [None req-38eca5a4-40a4-4366-b89e-5f373ca5e70f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 12 14:35:27.968936 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: warnings.warn(msg) Oct 12 14:35:27.968936 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: Oct 12 14:35:27.969817 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eca5a4-40a4-4366-b89e-5f373ca5e70f admin admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:35:27.981838 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 589/1177] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Oct 12 14:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 14:35:27.986210 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2294e-c533-4cf3-b4bb-5b4afbdb8d4a admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:35:27.987793 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-16d2294e-c533-4cf3-b4bb-5b4afbdb8d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:35:28.021469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2294e-c533-4cf3-b4bb-5b4afbdb8d4a admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:35:28.113469 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a51a0b-36df-43bb-8479-f0f8b31e6c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:35:28.114129 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a51a0b-36df-43bb-8479-f0f8b31e6c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:28.114377 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a51a0b-36df-43bb-8479-f0f8b31e6c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:28.114642 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a51a0b-36df-43bb-8479-f0f8b31e6c81 admin admin] PATH_INFO: `/v3/projects/bb629790291846a5b0caace80213a448/users/ff21448d426f425d963d190a2cc629cb/roles/6d1894444bfd400a89591929bd8925de` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:28.137790 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a51a0b-36df-43bb-8479-f0f8b31e6c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb629790291846a5b0caace80213a448, user_id=ff21448d426f425d963d190a2cc629cb, role_id=6d1894444bfd400a89591929bd8925de)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:35:28.141116 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a51a0b-36df-43bb-8479-f0f8b31e6c81 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:35:28.165488 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 589/1178] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Oct 12 14:35:27 2022] PUT /identity/v3/projects/bb629790291846a5b0caace80213a448/users/ff21448d426f425d963d190a2cc629cb/roles/6d1894444bfd400a89591929bd8925de => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 14:35:28.176660 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df1a9b-fed6-46c5-9fbb-0db70bc23dcc None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:28.176945 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df1a9b-fed6-46c5-9fbb-0db70bc23dcc None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:28.177218 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df1a9b-fed6-46c5-9fbb-0db70bc23dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:28.235286 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-76df1a9b-fed6-46c5-9fbb-0db70bc23dcc None None] MFA Rules not processed for user `ff21448d426f425d963d190a2cc629cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:35:28.236414 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-76df1a9b-fed6-46c5-9fbb-0db70bc23dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:35:28.314448 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 590/1179] 10.222.0.16 () {58 vars in 958 bytes} [Wed Oct 12 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:35:28.327623 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a25b9a-269e-4cd7-8764-83c7eebee60f None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:28.327875 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a25b9a-269e-4cd7-8764-83c7eebee60f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:28.328142 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a25b9a-269e-4cd7-8764-83c7eebee60f None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:28.329499 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 590/1180] 10.222.0.16 () {54 vars in 963 bytes} [Wed Oct 12 14:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:35:28.334401 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf98e50-0511-45fd-a2f4-b297c0208d51 None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:28.334683 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf98e50-0511-45fd-a2f4-b297c0208d51 None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:28.334952 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf98e50-0511-45fd-a2f4-b297c0208d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:28.367235 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:35:28.411385 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-2cf98e50-0511-45fd-a2f4-b297c0208d51 None None] MFA Rules not processed for user `c8514f660bf949ada15e8d7fc6a062a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:35:28.412574 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-2cf98e50-0511-45fd-a2f4-b297c0208d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:35:28.485715 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 591/1181] 10.222.0.16 () {58 vars in 1060 bytes} [Wed Oct 12 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:35:28.493487 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a25b9a-269e-4cd7-8764-83c7eebee60f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:35:28.494999 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-79a25b9a-269e-4cd7-8764-83c7eebee60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:35:28.544177 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:35:28.599302 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a25b9a-269e-4cd7-8764-83c7eebee60f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:35:28.674580 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981043d9-bfcc-4c70-94db-2106188c810f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8514f660bf949ada15e8d7fc6a062a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:35:28.675203 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981043d9-bfcc-4c70-94db-2106188c810f service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:28.675483 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981043d9-bfcc-4c70-94db-2106188c810f service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:28.675745 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981043d9-bfcc-4c70-94db-2106188c810f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:28.677309 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-981043d9-bfcc-4c70-94db-2106188c810f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:35:28.776322 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981043d9-bfcc-4c70-94db-2106188c810f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:35:28.777652 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981043d9-bfcc-4c70-94db-2106188c810f service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:35:28.781829 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:35:28.797288 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 591/1182] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:35:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:35:28.847393 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545387ea-0f9b-4870-b558-3906a8f46c6a None None] REQUEST_METHOD: `GET` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:28.847679 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545387ea-0f9b-4870-b558-3906a8f46c6a None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:28.847951 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545387ea-0f9b-4870-b558-3906a8f46c6a None None] PATH_INFO: `/` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:28.849232 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 592/1183] 10.222.0.16 () {52 vars in 830 bytes} [Wed Oct 12 14:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 14:35:28.852667 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981043d9-bfcc-4c70-94db-2106188c810f service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:35:28.853860 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-981043d9-bfcc-4c70-94db-2106188c810f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:35:28.950410 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981043d9-bfcc-4c70-94db-2106188c810f service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:35:29.042651 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd24bcb-513e-4978-b265-a82a41f055f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:35:29.043227 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd24bcb-513e-4978-b265-a82a41f055f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:29.043500 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd24bcb-513e-4978-b265-a82a41f055f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:29.043766 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd24bcb-513e-4978-b265-a82a41f055f5 admin admin] PATH_INFO: `/v3/users/ff21448d426f425d963d190a2cc629cb` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:29.057116 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd24bcb-513e-4978-b265-a82a41f055f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff21448d426f425d963d190a2cc629cb)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:35:29.058132 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: WARNING py.warnings [None req-1cd24bcb-513e-4978-b265-a82a41f055f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 14:35:29.058132 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: warnings.warn(msg) Oct 12 14:35:29.058132 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: Oct 12 14:35:29.059370 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd24bcb-513e-4978-b265-a82a41f055f5 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:35:29.068763 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:35:29.124989 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1cd24bcb-513e-4978-b265-a82a41f055f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff21448d426f425d963d190a2cc629cb'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 14:35:29.125602 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1cd24bcb-513e-4978-b265-a82a41f055f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff21448d426f425d963d190a2cc629cb'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 14:35:29.150117 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1cd24bcb-513e-4978-b265-a82a41f055f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff21448d426f425d963d190a2cc629cb'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 14:35:29.160408 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1cd24bcb-513e-4978-b265-a82a41f055f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff21448d426f425d963d190a2cc629cb'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 14:35:29.160917 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1cd24bcb-513e-4978-b265-a82a41f055f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff21448d426f425d963d190a2cc629cb'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 14:35:29.179991 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 592/1184] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Oct 12 14:35:28 2022] DELETE /identity/v3/users/ff21448d426f425d963d190a2cc629cb => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:35:29.184399 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbcbf8-605d-4e90-97c4-418f2756879e None None] REQUEST_METHOD: `POST` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:29.184903 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbcbf8-605d-4e90-97c4-418f2756879e None None] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:29.185008 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbcbf8-605d-4e90-97c4-418f2756879e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:29.241679 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.auth.core [None req-34fbcbf8-605d-4e90-97c4-418f2756879e None None] MFA Rules not processed for user `3ce73a2078a24b19af860a9906b4d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=81566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 14:35:29.242872 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-34fbcbf8-605d-4e90-97c4-418f2756879e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:35:29.319591 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=100 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:35:29.332448 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 593/1185] 10.222.0.16 () {58 vars in 958 bytes} [Wed Oct 12 14:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 14:35:29.342245 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd24bcb-513e-4978-b265-a82a41f055f5 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:35:29.343637 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-1cd24bcb-513e-4978-b265-a82a41f055f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:35:29.448162 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd24bcb-513e-4978-b265-a82a41f055f5 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:35:29.502283 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:35:29.522505 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf013572-0bf6-4ec4-a4ee-038f9835f623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0d533e36ff4a5186f8704480b58646', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '720b11237d2e48128c171be45fcd0ad0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:35:29.523110 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf013572-0bf6-4ec4-a4ee-038f9835f623 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:29.523414 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf013572-0bf6-4ec4-a4ee-038f9835f623 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:29.523691 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf013572-0bf6-4ec4-a4ee-038f9835f623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:29.525263 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-cf013572-0bf6-4ec4-a4ee-038f9835f623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:35:29.633474 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf013572-0bf6-4ec4-a4ee-038f9835f623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:35:29.634905 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf013572-0bf6-4ec4-a4ee-038f9835f623 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:35:29.653338 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 593/1186] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Oct 12 14:35:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 14:35:30.012680 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbcbf8-605d-4e90-97c4-418f2756879e None None] Authenticating user token {{(pid=81566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 14:35:30.013944 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.fernet_utils [None req-34fbcbf8-605d-4e90-97c4-418f2756879e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 14:35:30.025317 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbcbf8-605d-4e90-97c4-418f2756879e None None] Validating token access rules against request {{(pid=81566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 14:35:30.126436 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc574f8-b8eb-431d-9473-a4f400d2f8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce73a2078a24b19af860a9906b4d83e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '671bb1b187584e1b95d55373780c0171', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 14:35:30.126979 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc574f8-b8eb-431d-9473-a4f400d2f8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 14:35:30.127295 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc574f8-b8eb-431d-9473-a4f400d2f8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 14:35:30.127559 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc574f8-b8eb-431d-9473-a4f400d2f8d1 admin admin] PATH_INFO: `/v3/projects/bb629790291846a5b0caace80213a448` {{(pid=81566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 14:35:30.137485 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc574f8-b8eb-431d-9473-a4f400d2f8d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb629790291846a5b0caace80213a448)` {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 14:35:30.138494 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: WARNING py.warnings [None req-6fc574f8-b8eb-431d-9473-a4f400d2f8d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 14:35:30.138494 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: warnings.warn(msg) Oct 12 14:35:30.138494 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: Oct 12 14:35:30.139840 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc574f8-b8eb-431d-9473-a4f400d2f8d1 admin admin] RBAC: Authorization granted {{(pid=81566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 14:35:30.198914 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [None req-6fc574f8-b8eb-431d-9473-a4f400d2f8d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb629790291846a5b0caace80213a448', 'request_id': 'req-6fc574f8-b8eb-431d-9473-a4f400d2f8d1'} {{(pid=81566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 14:35:30.211888 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [None req-6fc574f8-b8eb-431d-9473-a4f400d2f8d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb629790291846a5b0caace80213a448', 'request_id': 'req-6fc574f8-b8eb-431d-9473-a4f400d2f8d1'} {{(pid=81566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 14:35:30.220286 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [None req-6fc574f8-b8eb-431d-9473-a4f400d2f8d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb629790291846a5b0caace80213a448', 'request_id': 'req-6fc574f8-b8eb-431d-9473-a4f400d2f8d1'} {{(pid=81566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 14:35:30.242557 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [None req-6fc574f8-b8eb-431d-9473-a4f400d2f8d1 admin admin] The token cache is being invalidate because project bb629790291846a5b0caace80213a448 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 14:35:30.242828 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [None req-6fc574f8-b8eb-431d-9473-a4f400d2f8d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 14:35:30.243371 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG keystone.notifications [None req-6fc574f8-b8eb-431d-9473-a4f400d2f8d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 14:35:30.244670 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: [pid: 81566|app: 0|req: 594/1187] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Oct 12 14:35:30 2022] DELETE /identity/v3/projects/bb629790291846a5b0caace80213a448 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 14:35:39.651036 dv-cinder-812685-17-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=62 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 14:35:40.240651 dv-cinder-812685-17-iscsi devstack@keystone.service[81566]: DEBUG dbcounter [-] [81566] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}